[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:01 dist-test-slave-zw7p krb5kdc[4800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:01 dist-test-slave-zw7p krb5kdc[4800](info): set up 2 sockets
Sep 27 07:09:01 dist-test-slave-zw7p krb5kdc[4800](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260927 07:09:01.946975 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:02.057834 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:42133
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:44405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:42133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:02.135437 4813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:02.135636 4813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:02.135665 4813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:02.136792 4813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:02.136831 4813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:02.136849 4813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:02.136866 4813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:02.136880 4813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:02.138329 4813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:44405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:42133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:42133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:02.138598 4813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:02.138808 4813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:02.143358 4821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:02.143446 4819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:02.143517 4818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:02.143638 4813 server_base.cc:1034] running on GCE node
I20260927 07:09:02.143879 4813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:02.144145 4813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:02.145287 4813 hybrid_clock.cc:648] HybridClock initialized: now 1790492942145274 us; error 26 us; skew 500 ppm
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:02.147643 4813 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:02.148072 4813 webserver.cc:460] Webserver started at http://127.4.174.126:34865/ using document root <none> and password file <none>
I20260927 07:09:02.148277 4813 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:02.148324 4813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:02.148447 4813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:02.149468 4813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "2fef77baecb34f48ba35f7445a1fc0eb"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "36686db9b73a27921960e615411a54a6"
server_key_iv: "a05e2797f114b53835ba833e2e91c5c1"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.149789 4813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "2fef77baecb34f48ba35f7445a1fc0eb"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "36686db9b73a27921960e615411a54a6"
server_key_iv: "a05e2797f114b53835ba833e2e91c5c1"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.151121 4813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:02.151829 4828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.151980 4813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:02.152057 4813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "2fef77baecb34f48ba35f7445a1fc0eb"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "36686db9b73a27921960e615411a54a6"
server_key_iv: "a05e2797f114b53835ba833e2e91c5c1"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.152113 4813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:02.165372 4813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:02.167490 4813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:02.167622 4813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:02.171314 4813 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:42133
I20260927 07:09:02.171350 4880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:42133 every 8 connection(s)
I20260927 07:09:02.171680 4813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:02.172056 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 4813
I20260927 07:09:02.172168 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:02.172676 4881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:02.172988 4793 external_mini_cluster.cc:1428] Setting key 1c4247939d100db8334acc3f6b307e8c
I20260927 07:09:02.175659 4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb: Bootstrap starting.
I20260927 07:09:02.176424 4881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:02.176721 4881 log.cc:824] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:02.177433 4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb: No bootstrap required, opened a new log
I20260927 07:09:02.179822 4881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fef77baecb34f48ba35f7445a1fc0eb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42133 } }
I20260927 07:09:02.179958 4881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:02.179991 4881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fef77baecb34f48ba35f7445a1fc0eb, State: Initialized, Role: FOLLOWER
I20260927 07:09:02.180114 4881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fef77baecb34f48ba35f7445a1fc0eb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42133 } }
I20260927 07:09:02.180186 4881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:02.180215 4881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:02.180259 4881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:02.180902 4881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fef77baecb34f48ba35f7445a1fc0eb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42133 } }
I20260927 07:09:02.181013 4881 leader_election.cc:302] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fef77baecb34f48ba35f7445a1fc0eb; no voters:
I20260927 07:09:02.181167 4881 leader_election.cc:288] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:02.181201 4886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:02.181347 4886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [term 1 LEADER]: Becoming Leader. State: Replica: 2fef77baecb34f48ba35f7445a1fc0eb, State: Running, Role: LEADER
I20260927 07:09:02.181428 4881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:02.181478 4886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fef77baecb34f48ba35f7445a1fc0eb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42133 } }
I20260927 07:09:02.181840 4886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fef77baecb34f48ba35f7445a1fc0eb. Latest consensus state: current_term: 1 leader_uuid: "2fef77baecb34f48ba35f7445a1fc0eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fef77baecb34f48ba35f7445a1fc0eb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42133 } } }
I20260927 07:09:02.181887 4888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fef77baecb34f48ba35f7445a1fc0eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fef77baecb34f48ba35f7445a1fc0eb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42133 } } }
I20260927 07:09:02.182003 4886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:02.182067 4888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:02.182325 4895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:02.183028 4895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:02.184661 4895 catalog_manager.cc:1347] Generated new cluster ID: af8db4d73f56489881e68c6d280d2d35
I20260927 07:09:02.184711 4895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:02.189617 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.173976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.4.174.126:42133)
0927 07:09:02.175852 (+ 1876us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:02.175859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:02.175888 (+ 29us) server_negotiation.cc:407] Connection header received
0927 07:09:02.177125 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:02.177132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:02.177184 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:02.177306 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:02.179480 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.180463 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.181952 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.182589 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.182688 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.186437 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:02.186441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:02.186441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:02.186453 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:02.187978 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.188358 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.188361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.188362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.188450 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.188661 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.188662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.188663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.188892 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:02.189042 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:02.189299 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 07:09:02.189423 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1830,"thread_start_us":78,"threads_started":1}
I20260927 07:09:02.213250 4895 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:02.213780 4895 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:02.216037 4895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fef77baecb34f48ba35f7445a1fc0eb: Generated new TSK 0
I20260927 07:09:02.216197 4895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:02.243785 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42133
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:44405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:02.333455 4907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:02.333650 4907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:02.333671 4907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:02.334790 4907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:02.334824 4907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:02.334848 4907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:02.336210 4907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:44405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42133
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:02.336385 4907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:02.336556 4907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:02.341305 4912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:02.341320 4913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:02.341318 4915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:02.341373 4907 server_base.cc:1034] running on GCE node
I20260927 07:09:02.341740 4907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:02.341971 4907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:02.343115 4907 hybrid_clock.cc:648] HybridClock initialized: now 1790492942343099 us; error 33 us; skew 500 ppm
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:02.345440 4907 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:02.345883 4907 webserver.cc:460] Webserver started at http://127.4.174.65:33215/ using document root <none> and password file <none>
I20260927 07:09:02.346078 4907 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:02.346125 4907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:02.346243 4907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:02.347308 4907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "8a0f939b451b4d42b1aafe43532d56a4"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "7d73514d4227d82383cd508d3815d870"
server_key_iv: "5510aedcda4a9a89a20d42c3b1e5ed64"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.347636 4907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "8a0f939b451b4d42b1aafe43532d56a4"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "7d73514d4227d82383cd508d3815d870"
server_key_iv: "5510aedcda4a9a89a20d42c3b1e5ed64"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.348855 4907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 07:09:02.349493 4922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.349632 4907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:02.349711 4907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "8a0f939b451b4d42b1aafe43532d56a4"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "7d73514d4227d82383cd508d3815d870"
server_key_iv: "5510aedcda4a9a89a20d42c3b1e5ed64"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.349769 4907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:02.362090 4907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:02.364182 4907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:02.364319 4907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:02.364526 4907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:02.364820 4907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:02.364852 4907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.364892 4907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:02.364916 4907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.370357 4907 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:45579
I20260927 07:09:02.370419 5035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:45579 every 8 connection(s)
I20260927 07:09:02.370751 4907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:02.378892 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 4907
I20260927 07:09:02.378979 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:02.379122 4793 external_mini_cluster.cc:1428] Setting key 57597b67680df209a9e77aa7123ff25a
I20260927 07:09:02.379403 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.371472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:37813 (local address 127.4.174.126:42133)
0927 07:09:02.371527 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:02.371529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:02.371720 (+ 191us) server_negotiation.cc:407] Connection header received
0927 07:09:02.372178 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:02.372179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:02.372188 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:02.372258 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:02.373233 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.373733 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.374736 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.374944 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.374998 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.376994 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:02.376995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:02.376996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:02.377003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:02.378080 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.378432 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.378434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.378434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.378487 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.378754 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.378755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.378755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.378867 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:02.378922 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:02.379209 (+ 287us) server_negotiation.cc:299] Negotiation successful
0927 07:09:02.379312 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 07:09:02.379554 5038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.371438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42133 (local address 127.4.174.65:37813)
0927 07:09:02.371601 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:02.371642 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:02.372103 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:02.372328 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:02.372332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:02.372923 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:02.373117 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:02.373123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.373804 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.373804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:02.374660 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:02.374663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.375004 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.375005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:02.375100 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.375558 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:02.375567 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:02.376908 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:02.378152 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:02.378159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:02.378162 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:02.378368 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:02.378547 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:02.378547 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:02.378548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:02.378670 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:02.378919 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:02.378925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:02.379082 (+ 157us) client_negotiation.cc:769] Sending connection context
0927 07:09:02.379181 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 07:09:02.379334 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260927 07:09:02.380059 5036 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42133
I20260927 07:09:02.380148 5036 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:02.380322 5036 heartbeater.cc:511] Master 127.4.174.126:42133 requested a full tablet report, sending...
I20260927 07:09:02.380887 4845 ts_manager.cc:194] Registered new tserver with Master: 8a0f939b451b4d42b1aafe43532d56a4 (127.4.174.65:45579)
I20260927 07:09:02.381778 4845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:37813
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:02.401715 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42133
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:44405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:02.479749 5041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:02.479938 5041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:02.479960 5041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:02.481105 5041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:02.481138 5041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:02.481163 5041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:02.482751 5041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:44405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42133
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:02.482968 5041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:02.483173 5041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:02.487805 5049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:02.487895 5041 server_base.cc:1034] running on GCE node
W20260927 07:09:02.487841 5047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:02.487814 5046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:02.488252 5041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:02.488499 5041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:02.489645 5041 hybrid_clock.cc:648] HybridClock initialized: now 1790492942489631 us; error 26 us; skew 500 ppm
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:02.491927 5041 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:02.492302 5041 webserver.cc:460] Webserver started at http://127.4.174.66:45415/ using document root <none> and password file <none>
I20260927 07:09:02.492506 5041 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:02.492552 5041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:02.492673 5041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:02.493670 5041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "21c97a93346e4f1c930d4f8cf5abb1e4"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "fe00dec1420e37c6b7e8839616196e6f"
server_key_iv: "31bf411f6a92212395e1af1820e7388e"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.494005 5041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "21c97a93346e4f1c930d4f8cf5abb1e4"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "fe00dec1420e37c6b7e8839616196e6f"
server_key_iv: "31bf411f6a92212395e1af1820e7388e"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.495239 5041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:02.495901 5056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.496044 5041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:02.496122 5041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "21c97a93346e4f1c930d4f8cf5abb1e4"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "fe00dec1420e37c6b7e8839616196e6f"
server_key_iv: "31bf411f6a92212395e1af1820e7388e"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.496181 5041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:02.505631 5041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:02.507323 5041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:02.507447 5041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:02.507628 5041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:02.507910 5041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:02.507944 5041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.507972 5041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:02.507994 5041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.513438 5041 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:44297
I20260927 07:09:02.513489 5169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:44297 every 8 connection(s)
I20260927 07:09:02.513780 5041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:02.515729 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 5041
I20260927 07:09:02.515802 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:02.515939 4793 external_mini_cluster.cc:1428] Setting key d42af4eb68241dec9dc2a9bc3c334445
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:02.522758 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.514422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:36619 (local address 127.4.174.126:42133)
0927 07:09:02.514477 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:02.514478 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:02.514702 (+ 224us) server_negotiation.cc:407] Connection header received
0927 07:09:02.515196 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:02.515197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:02.515209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:02.515262 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:02.516258 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.516793 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.517909 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.518100 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.518151 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.520348 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:02.520350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:02.520350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:02.520367 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:02.521563 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.521906 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.521907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.521907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.521959 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.522160 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.522161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.522161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.522265 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:02.522320 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:02.522588 (+ 268us) server_negotiation.cc:299] Negotiation successful
0927 07:09:02.522668 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 07:09:02.522934 5172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.514392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42133 (local address 127.4.174.66:36619)
0927 07:09:02.514578 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:02.514622 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:02.515105 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:02.515349 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:02.515354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:02.515942 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:02.516178 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:02.516184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.516867 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.516868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:02.517815 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:02.517817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.518164 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.518165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:02.518303 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.518829 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:02.518837 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:02.520266 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:02.521633 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:02.521638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:02.521640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:02.521836 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:02.522010 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:02.522011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:02.522011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:02.522108 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:02.522318 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:02.522324 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:02.522452 (+ 128us) client_negotiation.cc:769] Sending connection context
0927 07:09:02.522538 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 07:09:02.522715 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
I20260927 07:09:02.523337 5170 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42133
I20260927 07:09:02.523435 5170 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:02.523587 5170 heartbeater.cc:511] Master 127.4.174.126:42133 requested a full tablet report, sending...
I20260927 07:09:02.523936 4845 ts_manager.cc:194] Registered new tserver with Master: 21c97a93346e4f1c930d4f8cf5abb1e4 (127.4.174.66:44297)
I20260927 07:09:02.524339 4845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:36619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:02.539145 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42133
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:44405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:02.617728 5175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:02.617964 5175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:02.617996 5175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:02.619189 5175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:02.619231 5175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:02.619266 5175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:02.620729 5175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:44405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42133
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:02.620972 5175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:02.621186 5175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:02.625741 5183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:02.625973 5175 server_base.cc:1034] running on GCE node
W20260927 07:09:02.625774 5180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:02.625801 5181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:02.626257 5175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:02.626475 5175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:02.627604 5175 hybrid_clock.cc:648] HybridClock initialized: now 1790492942627598 us; error 45 us; skew 500 ppm
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:02.629882 5175 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:02.630293 5175 webserver.cc:460] Webserver started at http://127.4.174.67:36597/ using document root <none> and password file <none>
I20260927 07:09:02.630474 5175 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:02.630511 5175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:02.630645 5175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:02.631639 5175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "29b1ddbfef1b4911b31ddadfb01f494f"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "148f0a7c9df222f471e596cca1177fc5"
server_key_iv: "4946f5fba8d02a4aff7876f1c36df894"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.631948 5175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "29b1ddbfef1b4911b31ddadfb01f494f"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "148f0a7c9df222f471e596cca1177fc5"
server_key_iv: "4946f5fba8d02a4aff7876f1c36df894"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.633159 5175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:02.633810 5190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.633962 5175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:02.634044 5175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "29b1ddbfef1b4911b31ddadfb01f494f"
format_stamp: "Formatted at 2026-09-27 07:09:02 on dist-test-slave-zw7p"
server_key: "148f0a7c9df222f471e596cca1177fc5"
server_key_iv: "4946f5fba8d02a4aff7876f1c36df894"
server_key_version: "encryptionkey@0"
I20260927 07:09:02.634097 5175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:02.654533 5175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:02.656759 5175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:02.656879 5175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:02.657065 5175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:02.657325 5175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:02.657361 5175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.657382 5175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:02.657395 5175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:02.662855 5175 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:42417
I20260927 07:09:02.662911 5303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:42417 every 8 connection(s)
I20260927 07:09:02.663197 5175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:02.663414 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 5175
I20260927 07:09:02.663502 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:02.663676 4793 external_mini_cluster.cc:1428] Setting key 3ea52056b7d808de5bcfbce68b3d55ef
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:02.672077 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.664124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:53627 (local address 127.4.174.126:42133)
0927 07:09:02.664171 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:02.664173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:02.664395 (+ 222us) server_negotiation.cc:407] Connection header received
0927 07:09:02.664914 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:02.664915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:02.664927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:02.664972 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:02.665887 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.666381 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.667383 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.667572 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.667627 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.669621 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:02.669622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:02.669623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:02.669630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:02.670828 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.671194 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.671195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.671196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.671250 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.671465 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.671466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.671466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.671581 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:02.671638 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:02.671901 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 07:09:02.671983 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:02.672269 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.664111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42133 (local address 127.4.174.67:53627)
0927 07:09:02.664279 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:02.664321 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:02.664808 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:02.665049 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:02.665052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:02.665608 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:02.665798 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:02.665803 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.666462 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.666462 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:02.667299 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:02.667302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.667639 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.667640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:02.667756 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.668233 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:02.668242 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:02.669537 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:02.670902 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:02.670910 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:02.670912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:02.671129 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:02.671302 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:02.671303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:02.671304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:02.671410 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:02.671636 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:02.671643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:02.671780 (+ 137us) client_negotiation.cc:769] Sending connection context
0927 07:09:02.671879 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 07:09:02.672044 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260927 07:09:02.672821 5304 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42133
I20260927 07:09:02.672926 5304 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:02.673080 5304 heartbeater.cc:511] Master 127.4.174.126:42133 requested a full tablet report, sending...
I20260927 07:09:02.673406 4845 ts_manager.cc:194] Registered new tserver with Master: 29b1ddbfef1b4911b31ddadfb01f494f (127.4.174.67:42417)
I20260927 07:09:02.673760 4845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:53627
I20260927 07:09:02.675552 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:02.697127 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.689922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.4.174.65:45579)
0927 07:09:02.690092 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:02.690097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:02.690107 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:02.690133 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:02.690134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:02.690149 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:02.690213 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:02.690747 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.691460 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.692200 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.692505 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.692555 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.694806 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:02.694813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:02.694813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:02.694820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:02.696097 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.696418 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.696421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.696421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.696473 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.696669 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.696670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.696670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.696779 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:02.696901 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:02.696980 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 07:09:02.697038 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
W20260927 07:09:02.697496 4950 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36554
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[4800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:02.716873 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:02.711048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36558 (local address 127.4.174.65:45579)
0927 07:09:02.711111 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:02.711113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:02.711173 (+ 60us) server_negotiation.cc:407] Connection header received
0927 07:09:02.711240 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:02.711241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:02.711249 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:02.711292 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:02.711756 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.712214 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.712887 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:02.713073 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:02.713120 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:02.714983 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:02.714984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:02.714985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:02.714990 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:02.716017 (+ 1027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.716319 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.716320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.716320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.716375 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:02.716540 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:02.716541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:02.716541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:02.716616 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:02.716687 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:02.716751 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 07:09:02.716806 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 07:09:02.717339 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 4907
I20260927 07:09:02.721083 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 5041
I20260927 07:09:02.724752 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 5175
I20260927 07:09:02.728822 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 4813
2026-09-27T07:09:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1738 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[5318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[5318](info): set up 2 sockets
Sep 27 07:09:02 dist-test-slave-zw7p krb5kdc[5318](info): commencing operation
krb5kdc: starting...
W20260927 07:09:03.677840 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:03 dist-test-slave-zw7p krb5kdc[5318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:03.789534 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:42049
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:42049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:03.867343 5331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:03.867556 5331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:03.867588 5331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:03.868711 5331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:03.868754 5331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:03.868773 5331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:03.868789 5331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:03.868803 5331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:03.870234 5331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:42049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:42049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:03.870469 5331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:03.870762 5331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:03.875262 5339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:03.875349 5331 server_base.cc:1034] running on GCE node
W20260927 07:09:03.875334 5337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:03.875290 5336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:03.875690 5331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:03.875959 5331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:03.877110 5331 hybrid_clock.cc:648] HybridClock initialized: now 1790492943877091 us; error 31 us; skew 500 ppm
Sep 27 07:09:03 dist-test-slave-zw7p krb5kdc[5318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492943, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:03.879468 5331 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:03.879894 5331 webserver.cc:460] Webserver started at http://127.4.174.126:38575/ using document root <none> and password file <none>
I20260927 07:09:03.880091 5331 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:03.880138 5331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:03.880255 5331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:03.881286 5331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "bd60e0866df5454389e775601da1df97"
format_stamp: "Formatted at 2026-09-27 07:09:03 on dist-test-slave-zw7p"
server_key: "67d61e5e0b09f3bc08e459c031a6fad0"
server_key_iv: "e369edfb030b1fba00e02ad8d01eb332"
server_key_version: "encryptionkey@0"
I20260927 07:09:03.881608 5331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "bd60e0866df5454389e775601da1df97"
format_stamp: "Formatted at 2026-09-27 07:09:03 on dist-test-slave-zw7p"
server_key: "67d61e5e0b09f3bc08e459c031a6fad0"
server_key_iv: "e369edfb030b1fba00e02ad8d01eb332"
server_key_version: "encryptionkey@0"
I20260927 07:09:03.882941 5331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:03.883690 5346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:03.883863 5331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:03.883944 5331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "bd60e0866df5454389e775601da1df97"
format_stamp: "Formatted at 2026-09-27 07:09:03 on dist-test-slave-zw7p"
server_key: "67d61e5e0b09f3bc08e459c031a6fad0"
server_key_iv: "e369edfb030b1fba00e02ad8d01eb332"
server_key_version: "encryptionkey@0"
I20260927 07:09:03.884003 5331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:03.895385 5331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:03.897408 5331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:03.897537 5331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:03.901073 5331 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:42049
I20260927 07:09:03.901110 5398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:42049 every 8 connection(s)
I20260927 07:09:03.901408 5331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:03.902170 5399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:03.903542 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 5331
I20260927 07:09:03.903607 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:03.903740 4793 external_mini_cluster.cc:1428] Setting key 4dfc34742123d99622ce73ea1b8cd0fa
I20260927 07:09:03.904604 5399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97: Bootstrap starting.
I20260927 07:09:03.905282 5399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:03.905560 5399 log.cc:824] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:03.906229 5399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97: No bootstrap required, opened a new log
I20260927 07:09:03.908360 5399 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60e0866df5454389e775601da1df97" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42049 } }
I20260927 07:09:03.908524 5399 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:03.908560 5399 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd60e0866df5454389e775601da1df97, State: Initialized, Role: FOLLOWER
I20260927 07:09:03.908682 5399 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60e0866df5454389e775601da1df97" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42049 } }
I20260927 07:09:03.908766 5399 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:03.908805 5399 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:03.908852 5399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:03.909390 5399 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60e0866df5454389e775601da1df97" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42049 } }
I20260927 07:09:03.909476 5399 leader_election.cc:302] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd60e0866df5454389e775601da1df97; no voters:
I20260927 07:09:03.909632 5399 leader_election.cc:288] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:03.909658 5404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:03.909749 5404 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [term 1 LEADER]: Becoming Leader. State: Replica: bd60e0866df5454389e775601da1df97, State: Running, Role: LEADER
I20260927 07:09:03.909835 5404 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60e0866df5454389e775601da1df97" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42049 } }
I20260927 07:09:03.909878 5399 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:03.910126 5404 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd60e0866df5454389e775601da1df97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60e0866df5454389e775601da1df97" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42049 } } }
I20260927 07:09:03.910182 5404 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:03.910306 5404 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd60e0866df5454389e775601da1df97. Latest consensus state: current_term: 1 leader_uuid: "bd60e0866df5454389e775601da1df97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60e0866df5454389e775601da1df97" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42049 } } }
I20260927 07:09:03.910353 5404 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97 [sys.catalog]: This master's current role is: LEADER
Sep 27 07:09:03 dist-test-slave-zw7p krb5kdc[5318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
W20260927 07:09:03.911484 5420 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:09:03.911542 5420 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:09:03.911638 5409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:03.911859 5409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:03.913470 5409 catalog_manager.cc:1347] Generated new cluster ID: bc2ff6050abb4d75b2b4fe51d65ceb3d
I20260927 07:09:03.913522 5409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:03.914193 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:03.904250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.4.174.126:42049)
0927 07:09:03.904420 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:03.904429 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:03.904460 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:09:03.904966 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:03.904975 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:03.905037 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:03.905181 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:03.905754 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:03.907106 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:03.907833 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:03.908459 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:03.908563 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:03.911015 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:03.911019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:03.911019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:03.911031 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:03.912745 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:03.913106 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:03.913109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:03.913110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:03.913194 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:03.913384 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:03.913385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:03.913386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:03.913655 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:03.913801 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:03.913904 (+ 103us) server_negotiation.cc:299] Negotiation successful
0927 07:09:03.914026 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260927 07:09:03.935555 5409 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:03.936260 5409 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:03.947081 5409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd60e0866df5454389e775601da1df97: Generated new TSK 0
I20260927 07:09:03.947283 5409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:03.971480 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42049
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:04.049453 5425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:04.049654 5425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:04.049675 5425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:04.050833 5425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:04.050870 5425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:04.050895 5425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:04.052330 5425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42049
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:04.052556 5425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:04.052763 5425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:04.057458 5433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:04.057454 5430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:04.057454 5431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:04.057471 5425 server_base.cc:1034] running on GCE node
I20260927 07:09:04.057863 5425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:04.058077 5425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:04.059212 5425 hybrid_clock.cc:648] HybridClock initialized: now 1790492944059192 us; error 32 us; skew 500 ppm
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:04.061605 5425 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:04.062021 5425 webserver.cc:460] Webserver started at http://127.4.174.65:35589/ using document root <none> and password file <none>
I20260927 07:09:04.062212 5425 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:04.062258 5425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:04.062382 5425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:04.063421 5425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "8ffd053e49534738a89991ab2921f7f3"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "857d73d99958bf529b128f06ef3a1366"
server_key_iv: "92038b4669f0b324bf9f639d3e6bb2eb"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.063745 5425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "8ffd053e49534738a89991ab2921f7f3"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "857d73d99958bf529b128f06ef3a1366"
server_key_iv: "92038b4669f0b324bf9f639d3e6bb2eb"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.064950 5425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:04.065609 5440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.065774 5425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:04.065853 5425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "8ffd053e49534738a89991ab2921f7f3"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "857d73d99958bf529b128f06ef3a1366"
server_key_iv: "92038b4669f0b324bf9f639d3e6bb2eb"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.065917 5425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:04.081120 5425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:04.083144 5425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:04.083276 5425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:04.083479 5425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:04.083766 5425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:04.083798 5425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.083827 5425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:04.083851 5425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.089234 5425 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:35631
I20260927 07:09:04.089308 5553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:35631 every 8 connection(s)
I20260927 07:09:04.089579 5425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:04.095569 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 5425
I20260927 07:09:04.095657 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:04.095813 4793 external_mini_cluster.cc:1428] Setting key af5759f3b3729578b138a52cc510394c
I20260927 07:09:04.098315 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.090284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:54975 (local address 127.4.174.126:42049)
0927 07:09:04.090362 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.090364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.090496 (+ 132us) server_negotiation.cc:407] Connection header received
0927 07:09:04.091417 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.091418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.091427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.091476 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:04.092018 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.092515 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.093438 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.093619 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.093660 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.095563 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:04.095565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:04.095565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:04.095571 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:04.096935 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.097388 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.097389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.097390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.097438 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.097679 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.097680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.097681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.097813 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:04.097877 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.098122 (+ 245us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.098211 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 07:09:04.098387 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.090222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42049 (local address 127.4.174.65:54975)
0927 07:09:04.090375 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:04.090415 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:04.091334 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:04.091533 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:04.091536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:04.091775 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:04.091959 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.091963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.092572 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.092573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.093381 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.093383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.093666 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.093667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.093761 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.094215 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:04.094223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:04.095479 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:04.097016 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:04.097024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:04.097026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:04.097322 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:04.097489 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:04.097490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:04.097490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:04.097620 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:04.097886 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:04.097892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:04.098007 (+ 115us) client_negotiation.cc:769] Sending connection context
0927 07:09:04.098095 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 07:09:04.098224 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260927 07:09:04.098837 5554 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42049
I20260927 07:09:04.098927 5554 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:04.099088 5554 heartbeater.cc:511] Master 127.4.174.126:42049 requested a full tablet report, sending...
I20260927 07:09:04.099602 5363 ts_manager.cc:194] Registered new tserver with Master: 8ffd053e49534738a89991ab2921f7f3 (127.4.174.65:35631)
I20260927 07:09:04.100486 5363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:54975
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:04.118613 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42049
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:04.196337 5559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:04.196544 5559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:04.196575 5559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:04.197737 5559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:04.197777 5559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:04.197811 5559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:04.199301 5559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42049
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:04.199534 5559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:04.199740 5559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:04.204519 5564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:04.204509 5565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:04.204509 5567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:04.204599 5559 server_base.cc:1034] running on GCE node
I20260927 07:09:04.204916 5559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:04.205098 5559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:04.206233 5559 hybrid_clock.cc:648] HybridClock initialized: now 1790492944206218 us; error 26 us; skew 500 ppm
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:04.208529 5559 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:04.208951 5559 webserver.cc:460] Webserver started at http://127.4.174.66:37921/ using document root <none> and password file <none>
I20260927 07:09:04.209123 5559 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:04.209161 5559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:04.209259 5559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:04.210260 5559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "38e299947aa1474a8b1bdc136976737e"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "16c751203a2acc05409d435ced7fa271"
server_key_iv: "34371b229f8f8953e3a873dff4346c62"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.210623 5559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "38e299947aa1474a8b1bdc136976737e"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "16c751203a2acc05409d435ced7fa271"
server_key_iv: "34371b229f8f8953e3a873dff4346c62"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.211861 5559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:04.212505 5574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.212641 5559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:04.212728 5559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "38e299947aa1474a8b1bdc136976737e"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "16c751203a2acc05409d435ced7fa271"
server_key_iv: "34371b229f8f8953e3a873dff4346c62"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.212791 5559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:04.233150 5559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:04.234895 5559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:04.235021 5559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:04.235246 5559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:04.235560 5559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:04.235594 5559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.235615 5559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:04.235637 5559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.240957 5559 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:35763
I20260927 07:09:04.241035 5687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:35763 every 8 connection(s)
I20260927 07:09:04.241300 5559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:04.242758 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 5559
I20260927 07:09:04.242866 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:04.243041 4793 external_mini_cluster.cc:1428] Setting key 3ced7b0a1000e62f6ab76976c755885b
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
I20260927 07:09:04.250883 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.242032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:39093 (local address 127.4.174.126:42049)
0927 07:09:04.242114 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.242115 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.242271 (+ 156us) server_negotiation.cc:407] Connection header received
0927 07:09:04.243436 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.243437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.243447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.243512 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:04.244336 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.244831 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.245847 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.246049 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.246104 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.248254 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:04.248257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:04.248258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:04.248266 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:04.249545 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.249921 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.249923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.249923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.249980 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.250219 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.250221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.250221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.250371 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:04.250441 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.250717 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.250790 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 07:09:04.251052 5690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.241988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42049 (local address 127.4.174.66:39093)
0927 07:09:04.242144 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:04.242187 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:04.243313 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:04.243575 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:04.243578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:04.243955 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:04.244246 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.244253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.244923 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.244925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.245757 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.245760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.246121 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.246122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.246250 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.246809 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:04.246821 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:04.248154 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:04.249619 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:04.249627 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:04.249630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:04.249856 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:04.250033 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:04.250034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:04.250034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:04.250142 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:04.250440 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:04.250444 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:04.250603 (+ 159us) client_negotiation.cc:769] Sending connection context
0927 07:09:04.250696 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 07:09:04.250847 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260927 07:09:04.251551 5688 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42049
I20260927 07:09:04.251648 5688 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:04.251821 5688 heartbeater.cc:511] Master 127.4.174.126:42049 requested a full tablet report, sending...
I20260927 07:09:04.252178 5363 ts_manager.cc:194] Registered new tserver with Master: 38e299947aa1474a8b1bdc136976737e (127.4.174.66:35763)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
I20260927 07:09:04.252592 5363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:39093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:04.265062 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42049
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:04.343501 5693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:04.343745 5693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:04.343779 5693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:04.344928 5693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:04.344964 5693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:04.344990 5693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:04.346395 5693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42049
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:04.346617 5693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:04.346796 5693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:04.351528 5701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:04.351523 5698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:04.351549 5693 server_base.cc:1034] running on GCE node
W20260927 07:09:04.351524 5699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:04.351871 5693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:04.352103 5693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:04.353241 5693 hybrid_clock.cc:648] HybridClock initialized: now 1790492944353231 us; error 31 us; skew 500 ppm
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:04.355610 5693 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:04.356029 5693 webserver.cc:460] Webserver started at http://127.4.174.67:46267/ using document root <none> and password file <none>
I20260927 07:09:04.356221 5693 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:04.356268 5693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:04.356388 5693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:04.357421 5693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "b3743ad8c4664cb6a4999116075232c5"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "ff7ced80264bc9832902fdf9cd1ccd39"
server_key_iv: "8debe2f7d2066121bd69eb387a2e4458"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.357749 5693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "b3743ad8c4664cb6a4999116075232c5"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "ff7ced80264bc9832902fdf9cd1ccd39"
server_key_iv: "8debe2f7d2066121bd69eb387a2e4458"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.359037 5693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:04.359779 5708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.359943 5693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 07:09:04.360030 5693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "b3743ad8c4664cb6a4999116075232c5"
format_stamp: "Formatted at 2026-09-27 07:09:04 on dist-test-slave-zw7p"
server_key: "ff7ced80264bc9832902fdf9cd1ccd39"
server_key_iv: "8debe2f7d2066121bd69eb387a2e4458"
server_key_version: "encryptionkey@0"
I20260927 07:09:04.360091 5693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:04.367030 5693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:04.368945 5693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:04.369086 5693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:04.369262 5693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:04.369495 5693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:04.369524 5693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.369544 5693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:04.369558 5693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:04.374864 5693 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:38427
I20260927 07:09:04.374940 5821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:38427 every 8 connection(s)
I20260927 07:09:04.375212 5693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:04.379243 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 5693
I20260927 07:09:04.379333 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:04.379472 4793 external_mini_cluster.cc:1428] Setting key d556c7aa0c61e3a90328d7d3e736e713
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:04.384017 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.375875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:38287 (local address 127.4.174.126:42049)
0927 07:09:04.375923 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.375925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.376091 (+ 166us) server_negotiation.cc:407] Connection header received
0927 07:09:04.377003 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.377005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.377018 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.377065 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:04.377654 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.378151 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.379167 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.379401 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.379467 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.381673 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:04.381675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:04.381675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:04.381681 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:04.382935 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.383295 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.383296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.383296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.383345 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.383515 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.383516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.383516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.383622 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:04.383664 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.383902 (+ 238us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.383954 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:09:04.384196 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.375839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42049 (local address 127.4.174.67:38287)
0927 07:09:04.375978 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:04.376018 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:04.376905 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:04.377143 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:04.377147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:04.377385 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:04.377569 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.377573 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.378228 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.378229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.379091 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.379093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.379474 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.379475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.379601 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.380104 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:04.380115 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:04.381594 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:04.383007 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:04.383013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:04.383014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:04.383238 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:04.383386 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:04.383386 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:04.383386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:04.383475 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:04.383670 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:04.383675 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:04.383806 (+ 131us) client_negotiation.cc:769] Sending connection context
0927 07:09:04.383887 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 07:09:04.383998 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260927 07:09:04.384567 5822 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42049
I20260927 07:09:04.384651 5822 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:04.384809 5822 heartbeater.cc:511] Master 127.4.174.126:42049 requested a full tablet report, sending...
I20260927 07:09:04.385109 5363 ts_manager.cc:194] Registered new tserver with Master: b3743ad8c4664cb6a4999116075232c5 (127.4.174.67:38427)
I20260927 07:09:04.385466 5363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:38287
I20260927 07:09:04.391080 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:09:04.397989 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.392712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59892 (local address 127.4.174.126:42049)
0927 07:09:04.392769 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.392771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.392807 (+ 36us) server_negotiation.cc:407] Connection header received
0927 07:09:04.392874 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.392875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.392885 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.392928 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:04.393453 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.393947 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.394682 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.394876 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.394927 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.395994 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:04.395995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:04.395996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:04.396003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:04.396940 (+ 937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.397221 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.397222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.397222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.397291 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.397492 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.397492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.397493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.397630 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:04.397714 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.397834 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.397903 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:04.400007 5363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:04.401578 5363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:04.407651 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.403814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46170 (local address 127.4.174.67:38427)
0927 07:09:04.404140 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.404144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.404237 (+ 93us) server_negotiation.cc:407] Connection header received
0927 07:09:04.404279 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.404281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.404296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.404362 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:04.404541 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.405311 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.406641 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.407279 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.407311 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.407350 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.407540 (+ 190us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.407578 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":77,"threads_started":1}
I20260927 07:09:04.407634 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.403634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:38427 (local address 127.0.0.1:46170)
0927 07:09:04.404171 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:04.404178 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:04.404219 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:04.404374 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:04.404377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:04.404380 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:04.404459 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.404461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.405401 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.405401 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.406532 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.406534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.407393 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.407394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.407448 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.407456 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:09:04.407508 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 07:09:04.407558 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":7,"thread_start_us":70,"threads_started":1}
I20260927 07:09:04.408046 5831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.403634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:35631 (local address 127.0.0.1:57748)
0927 07:09:04.403976 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:04.404007 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:04.404076 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:04.404375 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:04.404378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:04.404384 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:04.404500 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.404503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.405822 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.405823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.406777 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.406779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.407818 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.407819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.407880 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.407884 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:04.407930 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 07:09:04.407970 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":44,"thread_start_us":75,"threads_started":1}
I20260927 07:09:04.408120 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.403956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.4.174.66:35763)
0927 07:09:04.404130 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.404135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.404320 (+ 185us) server_negotiation.cc:407] Connection header received
0927 07:09:04.404361 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.404363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.404376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.404439 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:04.404656 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.405717 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.407044 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.407711 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.407771 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.407821 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.408011 (+ 190us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.408049 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260927 07:09:04.408277 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.403838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:35763 (local address 127.0.0.1:55298)
0927 07:09:04.404205 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:04.404215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:04.404266 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:04.404544 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:04.404544 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:04.404546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:04.404585 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.404587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.405816 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.405817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.406954 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.406957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.407848 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.407849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.407904 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.407907 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:04.407946 (+ 39us) client_negotiation.cc:240] Negotiation successful
0927 07:09:04.407975 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":26,"thread_start_us":58,"threads_started":1}
I20260927 07:09:04.408332 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.403787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.4.174.65:35631)
0927 07:09:04.403962 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.403967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.404061 (+ 94us) server_negotiation.cc:407] Connection header received
0927 07:09:04.404175 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.404178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.404195 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.404281 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:04.404601 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.405709 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.406853 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.407524 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.407591 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.407642 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.408210 (+ 568us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.408260 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":67,"threads_started":1}
I20260927 07:09:04.408373 5756 tablet_service.cc:1467] Processing CreateTablet for tablet 6e04b5175c754870bbbba78cede2e8eb (DEFAULT_TABLE table=test-table [id=55b1a5f091214dac8bad62fd8fab15cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:04.408759 5756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e04b5175c754870bbbba78cede2e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:04.408944 5622 tablet_service.cc:1467] Processing CreateTablet for tablet 6e04b5175c754870bbbba78cede2e8eb (DEFAULT_TABLE table=test-table [id=55b1a5f091214dac8bad62fd8fab15cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:04.409068 5488 tablet_service.cc:1467] Processing CreateTablet for tablet 6e04b5175c754870bbbba78cede2e8eb (DEFAULT_TABLE table=test-table [id=55b1a5f091214dac8bad62fd8fab15cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:04.409271 5622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e04b5175c754870bbbba78cede2e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:04.409431 5488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e04b5175c754870bbbba78cede2e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:04.412496 5836 tablet_bootstrap.cc:492] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5: Bootstrap starting.
I20260927 07:09:04.412492 5838 tablet_bootstrap.cc:492] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e: Bootstrap starting.
I20260927 07:09:04.413041 5838 tablet_bootstrap.cc:654] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:04.413043 5836 tablet_bootstrap.cc:654] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:04.413470 5838 log.cc:824] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:04.413473 5836 log.cc:824] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:04.413777 5837 tablet_bootstrap.cc:492] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3: Bootstrap starting.
I20260927 07:09:04.414134 5838 tablet_bootstrap.cc:492] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e: No bootstrap required, opened a new log
I20260927 07:09:04.414194 5838 ts_tablet_manager.cc:1397] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:04.414345 5837 tablet_bootstrap.cc:654] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:04.414820 5837 log.cc:824] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:04.415320 5836 tablet_bootstrap.cc:492] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5: No bootstrap required, opened a new log
I20260927 07:09:04.415383 5836 ts_tablet_manager.cc:1397] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 07:09:04.415804 5838 raft_consensus.cc:348] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.415941 5838 raft_consensus.cc:374] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:04.415971 5838 raft_consensus.cc:729] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e299947aa1474a8b1bdc136976737e, State: Initialized, Role: FOLLOWER
I20260927 07:09:04.416087 5838 consensus_queue.cc:260] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.416344 5838 ts_tablet_manager.cc:1428] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:04.416378 5688 heartbeater.cc:503] Master 127.4.174.126:42049 was elected leader, sending a full tablet report...
I20260927 07:09:04.416857 5836 raft_consensus.cc:348] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.416988 5836 raft_consensus.cc:374] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:04.417019 5836 raft_consensus.cc:729] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3743ad8c4664cb6a4999116075232c5, State: Initialized, Role: FOLLOWER
I20260927 07:09:04.417129 5836 consensus_queue.cc:260] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.417382 5836 ts_tablet_manager.cc:1428] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:04.417377 5822 heartbeater.cc:503] Master 127.4.174.126:42049 was elected leader, sending a full tablet report...
I20260927 07:09:04.417757 5837 tablet_bootstrap.cc:492] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3: No bootstrap required, opened a new log
I20260927 07:09:04.417824 5837 ts_tablet_manager.cc:1397] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 07:09:04.419365 5837 raft_consensus.cc:348] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.419502 5837 raft_consensus.cc:374] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:04.419521 5837 raft_consensus.cc:729] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffd053e49534738a89991ab2921f7f3, State: Initialized, Role: FOLLOWER
I20260927 07:09:04.419605 5837 consensus_queue.cc:260] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.419824 5837 ts_tablet_manager.cc:1428] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:04.419883 5554 heartbeater.cc:503] Master 127.4.174.126:42049 was elected leader, sending a full tablet report...
W20260927 07:09:04.491817 5689 tablet.cc:2369] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:09:04.590298 5555 tablet.cc:2369] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:09:04.625754 5823 tablet.cc:2369] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:04.734695 5844 raft_consensus.cc:482] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:04.734848 5844 raft_consensus.cc:504] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.735205 5844 leader_election.cc:288] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3743ad8c4664cb6a4999116075232c5 (127.4.174.67:38427), 38e299947aa1474a8b1bdc136976737e (127.4.174.66:35763)
I20260927 07:09:04.739135 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.735414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:59197 (local address 127.4.174.66:35763)
0927 07:09:04.735480 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.735482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.735658 (+ 176us) server_negotiation.cc:407] Connection header received
0927 07:09:04.735708 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.735709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.735717 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.735767 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:04.735912 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.736648 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.738192 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.738767 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.738832 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.738841 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.739027 (+ 186us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.739062 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 07:09:04.739135 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.735539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:38241 (local address 127.4.174.67:38427)
0927 07:09:04.735594 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.735596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.735603 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:04.735632 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.735632 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.735642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.735702 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:04.735901 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.736652 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.738188 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.738760 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.738832 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.738841 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.739028 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.739062 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 07:09:04.739178 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.735357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:38427 (local address 127.4.174.65:38241)
0927 07:09:04.735467 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:04.735479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:04.735538 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:04.735752 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:04.735752 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:04.735755 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:04.735838 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.735840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.736750 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.736750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.738107 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.738109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.738861 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.738862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.738949 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.738954 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:04.739021 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 07:09:04.739072 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":66,"thread_start_us":62,"threads_started":1}
I20260927 07:09:04.739187 5845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.735367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:35763 (local address 127.4.174.65:59197)
0927 07:09:04.735601 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:04.735609 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:04.735652 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:04.735780 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:04.735781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:04.735782 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:04.735849 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.735850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.736745 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.736747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.738107 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:04.738109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.738859 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.738861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:04.738949 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.738955 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 07:09:04.739021 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 07:09:04.739072 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":65,"thread_start_us":56,"threads_started":1}
I20260927 07:09:04.739631 5776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e04b5175c754870bbbba78cede2e8eb" candidate_uuid: "8ffd053e49534738a89991ab2921f7f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3743ad8c4664cb6a4999116075232c5" is_pre_election: true
I20260927 07:09:04.739773 5776 raft_consensus.cc:2454] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ffd053e49534738a89991ab2921f7f3 in term 0.
I20260927 07:09:04.739760 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e04b5175c754870bbbba78cede2e8eb" candidate_uuid: "8ffd053e49534738a89991ab2921f7f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e299947aa1474a8b1bdc136976737e" is_pre_election: true
I20260927 07:09:04.739916 5642 raft_consensus.cc:2454] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ffd053e49534738a89991ab2921f7f3 in term 0.
I20260927 07:09:04.740001 5443 leader_election.cc:302] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffd053e49534738a89991ab2921f7f3, b3743ad8c4664cb6a4999116075232c5; no voters:
I20260927 07:09:04.740170 5844 raft_consensus.cc:2781] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:04.740207 5844 raft_consensus.cc:482] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:04.740221 5844 raft_consensus.cc:3037] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:04.740870 5844 raft_consensus.cc:504] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.740980 5844 leader_election.cc:288] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [CANDIDATE]: Term 1 election: Requested vote from peers b3743ad8c4664cb6a4999116075232c5 (127.4.174.67:38427), 38e299947aa1474a8b1bdc136976737e (127.4.174.66:35763)
I20260927 07:09:04.741127 5642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e04b5175c754870bbbba78cede2e8eb" candidate_uuid: "8ffd053e49534738a89991ab2921f7f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e299947aa1474a8b1bdc136976737e"
I20260927 07:09:04.741135 5776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e04b5175c754870bbbba78cede2e8eb" candidate_uuid: "8ffd053e49534738a89991ab2921f7f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3743ad8c4664cb6a4999116075232c5"
I20260927 07:09:04.741210 5642 raft_consensus.cc:3037] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:04.741210 5776 raft_consensus.cc:3037] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:04.742028 5776 raft_consensus.cc:2454] T 6e04b5175c754870bbbba78cede2e8eb P b3743ad8c4664cb6a4999116075232c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ffd053e49534738a89991ab2921f7f3 in term 1.
I20260927 07:09:04.742028 5642 raft_consensus.cc:2454] T 6e04b5175c754870bbbba78cede2e8eb P 38e299947aa1474a8b1bdc136976737e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ffd053e49534738a89991ab2921f7f3 in term 1.
I20260927 07:09:04.742190 5444 leader_election.cc:302] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38e299947aa1474a8b1bdc136976737e, 8ffd053e49534738a89991ab2921f7f3; no voters:
I20260927 07:09:04.742298 5844 raft_consensus.cc:2781] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:04.742429 5844 raft_consensus.cc:686] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [term 1 LEADER]: Becoming Leader. State: Replica: 8ffd053e49534738a89991ab2921f7f3, State: Running, Role: LEADER
I20260927 07:09:04.742522 5844 consensus_queue.cc:237] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } }
I20260927 07:09:04.743321 5363 catalog_manager.cc:5495] T 6e04b5175c754870bbbba78cede2e8eb P 8ffd053e49534738a89991ab2921f7f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ffd053e49534738a89991ab2921f7f3 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "8ffd053e49534738a89991ab2921f7f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffd053e49534738a89991ab2921f7f3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3743ad8c4664cb6a4999116075232c5" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38e299947aa1474a8b1bdc136976737e" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 35763 } health_report { overall_health: UNKNOWN } } }
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:04.764271 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.757600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.4.174.65:35631)
0927 07:09:04.757672 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.757674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.757764 (+ 90us) server_negotiation.cc:407] Connection header received
0927 07:09:04.757810 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.757811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.757819 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.757865 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:04.758353 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.758867 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.759546 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.759720 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.759778 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.761873 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:04.761883 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:04.761884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:04.761891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:04.763337 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.763644 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.763647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.763648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.763701 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.763842 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.763842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.763842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.763969 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:04.764095 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.764122 (+ 27us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.764189 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:04.784646 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.778475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.4.174.65:35631)
0927 07:09:04.778536 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.778537 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.778624 (+ 87us) server_negotiation.cc:407] Connection header received
0927 07:09:04.778718 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.778719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.778730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.778776 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:04.779276 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.779745 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.780456 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.780641 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.780693 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.782715 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:04.782716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:04.782716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:04.782723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:04.783794 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.784088 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.784088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.784089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.784132 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.784262 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.784262 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.784263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.784341 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:04.784409 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.784520 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.784576 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:04.804590 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:04.798861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.4.174.65:35631)
0927 07:09:04.798922 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:04.798923 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:04.798990 (+ 67us) server_negotiation.cc:407] Connection header received
0927 07:09:04.799033 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:04.799033 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:04.799042 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:04.799085 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:04.799524 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.799971 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.800656 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:04.800824 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:04.800864 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:04.802835 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:04.802837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:04.802837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:04.802843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:04.803841 (+ 998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.804094 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.804095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.804095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.804134 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:04.804256 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:04.804256 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:04.804257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:04.804331 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:04.804395 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:04.804471 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 07:09:04.804522 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260927 07:09:04.805039 5468 tablet_service.cc:2908] Rejecting scan request for tablet 6e04b5175c754870bbbba78cede2e8eb: Uninitialized: clean time has not yet been initialized
I20260927 07:09:04.805801 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 5425
I20260927 07:09:04.809830 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 5559
I20260927 07:09:04.813454 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 5693
I20260927 07:09:04.817169 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 5331
2026-09-27T07:09:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2088 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5861](info): set up 2 sockets
Sep 27 07:09:04 dist-test-slave-zw7p krb5kdc[5861](info): commencing operation
krb5kdc: starting...
W20260927 07:09:05.761874 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:05 dist-test-slave-zw7p krb5kdc[5861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:05.873865 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:33817
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:33817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:05.951952 5874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:05.952157 5874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:05.952186 5874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:05.953313 5874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:05.953351 5874 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 07:09:05.953362 5874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:05.953374 5874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:05.953384 5874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:05.953395 5874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:05.954864 5874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:33817
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:33817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:05.955056 5874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:05.955240 5874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:05.959944 5880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:05.959959 5882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:05.959976 5874 server_base.cc:1034] running on GCE node
W20260927 07:09:05.959959 5879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:05.960443 5874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:05.960718 5874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:05.961854 5874 hybrid_clock.cc:648] HybridClock initialized: now 1790492945961836 us; error 30 us; skew 500 ppm
Sep 27 07:09:05 dist-test-slave-zw7p krb5kdc[5861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492945, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:05.964264 5874 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:05.964679 5874 webserver.cc:460] Webserver started at http://127.4.174.126:41149/ using document root <none> and password file <none>
I20260927 07:09:05.964887 5874 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:05.964934 5874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:05.965054 5874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:05.966054 5874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "32b753d6afad46c8896e3201a7593266"
format_stamp: "Formatted at 2026-09-27 07:09:05 on dist-test-slave-zw7p"
server_key: "b7afee6e13660bcad5fd65545464279a"
server_key_iv: "943138e0762f09b1ef50225cd40892be"
server_key_version: "encryptionkey@0"
I20260927 07:09:05.966373 5874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "32b753d6afad46c8896e3201a7593266"
format_stamp: "Formatted at 2026-09-27 07:09:05 on dist-test-slave-zw7p"
server_key: "b7afee6e13660bcad5fd65545464279a"
server_key_iv: "943138e0762f09b1ef50225cd40892be"
server_key_version: "encryptionkey@0"
I20260927 07:09:05.967667 5874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:05.968341 5889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:05.968508 5874 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:05.968586 5874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "32b753d6afad46c8896e3201a7593266"
format_stamp: "Formatted at 2026-09-27 07:09:05 on dist-test-slave-zw7p"
server_key: "b7afee6e13660bcad5fd65545464279a"
server_key_iv: "943138e0762f09b1ef50225cd40892be"
server_key_version: "encryptionkey@0"
I20260927 07:09:05.968649 5874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:05.984262 5874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:05.986059 5874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:05.986197 5874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:05.990510 5874 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:33817
I20260927 07:09:05.990590 5951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:33817 every 8 connection(s)
I20260927 07:09:05.990877 5874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:05.991466 5952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:05.993678 5952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266: Bootstrap starting.
I20260927 07:09:05.994237 5952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:05.994508 5952 log.cc:824] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:05.995178 5952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266: No bootstrap required, opened a new log
I20260927 07:09:05.997238 5952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b753d6afad46c8896e3201a7593266" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33817 } }
I20260927 07:09:05.997385 5952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:05.997406 5952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b753d6afad46c8896e3201a7593266, State: Initialized, Role: FOLLOWER
I20260927 07:09:05.997532 5952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b753d6afad46c8896e3201a7593266" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33817 } }
I20260927 07:09:05.997602 5952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:05.997643 5952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:05.997690 5952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:05.998216 5952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b753d6afad46c8896e3201a7593266" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33817 } }
I20260927 07:09:05.998298 5952 leader_election.cc:302] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32b753d6afad46c8896e3201a7593266; no voters:
I20260927 07:09:05.998452 5952 leader_election.cc:288] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:05.998492 5955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:05.998507 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 5874
I20260927 07:09:05.998607 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:05.998646 5955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [term 1 LEADER]: Becoming Leader. State: Replica: 32b753d6afad46c8896e3201a7593266, State: Running, Role: LEADER
I20260927 07:09:05.998750 5952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:05.998772 4793 external_mini_cluster.cc:1428] Setting key 9d85c444394c21e0ffd74f7e7e4e0db0
I20260927 07:09:05.998730 5955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b753d6afad46c8896e3201a7593266" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33817 } }
I20260927 07:09:05.999074 5955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32b753d6afad46c8896e3201a7593266. Latest consensus state: current_term: 1 leader_uuid: "32b753d6afad46c8896e3201a7593266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b753d6afad46c8896e3201a7593266" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33817 } } }
I20260927 07:09:05.999153 5955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:05.999310 5956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32b753d6afad46c8896e3201a7593266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b753d6afad46c8896e3201a7593266" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33817 } } }
I20260927 07:09:05.999372 5956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:05.999501 5961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:06.000005 5961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:06.002432 5961 catalog_manager.cc:1347] Generated new cluster ID: c1ecbeb3154b43f8a7a48587ad53f547
I20260927 07:09:06.002480 5961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:06.010355 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:05.999360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46380 (local address 127.4.174.126:33817)
0927 07:09:05.999632 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:05.999642 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:05.999905 (+ 263us) server_negotiation.cc:407] Connection header received
0927 07:09:06.000030 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.000038 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.000099 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.000256 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.001197 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.002619 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.003672 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.004287 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.004404 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.007007 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.007011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.007011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.007022 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.008804 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.009196 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.009198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.009199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.009292 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.009474 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.009475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.009475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.009765 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.009893 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.010002 (+ 109us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.010124 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":81,"threads_started":1}
I20260927 07:09:06.017139 5961 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:06.017611 5961 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:06.020491 5961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266: Generated new TSK 0
I20260927 07:09:06.020648 5961 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:06.053340 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33817
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:06.142433 5978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:06.142685 5978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:06.142712 5978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:06.143906 5978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:06.143953 5978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:06.143977 5978 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:06.144012 5978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:06.145462 5978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:33817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:06.145712 5978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:06.145941 5978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 07:09:06.150926 5978 server_base.cc:1034] running on GCE node
W20260927 07:09:06.150993 5983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:06.151083 5984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:06.150993 5986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:06.151374 5978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:06.151629 5978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:06.152807 5978 hybrid_clock.cc:648] HybridClock initialized: now 1790492946152788 us; error 32 us; skew 500 ppm
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:06.161473 5978 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:06.161979 5978 webserver.cc:460] Webserver started at http://127.4.174.65:38863/ using document root <none> and password file <none>
I20260927 07:09:06.162211 5978 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:06.162261 5978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:06.162390 5978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:06.163534 5978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "81a3fd3d811a4418b3096c572445f1a4"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "fdef29f2d3eef5fa26c00e79e9b328b1"
server_key_iv: "d4c62d5b242693835777082d88c61247"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.163874 5978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "81a3fd3d811a4418b3096c572445f1a4"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "fdef29f2d3eef5fa26c00e79e9b328b1"
server_key_iv: "d4c62d5b242693835777082d88c61247"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.165182 5978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:06.165864 5993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.166050 5978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:06.166148 5978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "81a3fd3d811a4418b3096c572445f1a4"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "fdef29f2d3eef5fa26c00e79e9b328b1"
server_key_iv: "d4c62d5b242693835777082d88c61247"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.166221 5978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:06.174638 5978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:06.176833 5978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:06.176977 5978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:06.177546 5978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:06.177597 5978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.177627 5978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:06.177644 5978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.184142 5978 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:46779
I20260927 07:09:06.184228 6108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:46779 every 8 connection(s)
I20260927 07:09:06.184557 5978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:06.187448 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.178100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:50419 (local address 127.4.174.126:33817)
0927 07:09:06.178212 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.178214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.178484 (+ 270us) server_negotiation.cc:407] Connection header received
0927 07:09:06.179016 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.179018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.179032 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.179106 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.179960 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.180814 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.182316 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.182645 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.182725 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.185252 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.185253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.185254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.185260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.186339 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.186738 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.186738 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.186739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.186788 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.186951 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.186952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.186952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.187046 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.187089 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.187316 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.187383 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 07:09:06.187587 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.178008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33817 (local address 127.4.174.65:50419)
0927 07:09:06.178350 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.178393 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.178921 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.179188 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.179193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.179525 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:06.179841 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.179848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.180922 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.180923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.182217 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.182220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.182730 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.182731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.182856 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.183473 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:06.183484 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:06.185161 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:06.186402 (+ 1241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.186408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.186410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.186680 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.186827 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.186827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.186827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.186916 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.187085 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:06.187088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:06.187204 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.187281 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.187400 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":86,"threads_started":1}
I20260927 07:09:06.188060 6109 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33817
I20260927 07:09:06.188138 6109 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:06.188311 6109 heartbeater.cc:511] Master 127.4.174.126:33817 requested a full tablet report, sending...
I20260927 07:09:06.188526 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 5978
I20260927 07:09:06.188621 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:06.188812 4793 external_mini_cluster.cc:1428] Setting key d7c503d8f9c4dfd00cea2453c399029b
I20260927 07:09:06.188839 5905 ts_manager.cc:194] Registered new tserver with Master: 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779)
I20260927 07:09:06.189680 5905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:50419
I20260927 07:09:06.194774 6115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.189032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33817 (local address 127.4.174.65:60725)
0927 07:09:06.189153 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.189163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.189213 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.190023 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.190024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.190202 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:06.190257 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.190258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.190929 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.190930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.191438 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.191439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.191768 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.191769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.191842 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.192244 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:06.192248 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:06.192515 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:06.193881 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.193882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.193882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.194036 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.194197 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.194198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.194198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.194252 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.194449 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:06.194451 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:06.194497 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.194570 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.194668 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
I20260927 07:09:06.194793 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.189873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:60725 (local address 127.4.174.126:33817)
0927 07:09:06.189920 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.189925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.189932 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:06.189948 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.189948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.189958 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.190008 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.190324 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.190838 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.191503 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.191710 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.191765 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.192592 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.192594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.192594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.192601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.193812 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.194092 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.194093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.194093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.194143 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.194300 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.194300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.194301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.194396 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.194449 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.194631 (+ 182us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.194715 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:06.213172 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33817
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:06.291905 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:06.292102 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:06.292124 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:06.293206 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:06.293241 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:06.293259 6118 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:06.293275 6118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:06.294658 6118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:33817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:06.294847 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:06.295012 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:06.299661 6124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:06.299737 6126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:06.299929 6118 server_base.cc:1034] running on GCE node
W20260927 07:09:06.299937 6123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:06.300185 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:06.300415 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:06.301559 6118 hybrid_clock.cc:648] HybridClock initialized: now 1790492946301544 us; error 30 us; skew 500 ppm
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:06.303958 6118 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:06.304350 6118 webserver.cc:460] Webserver started at http://127.4.174.66:32833/ using document root <none> and password file <none>
I20260927 07:09:06.304556 6118 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:06.304603 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:06.304725 6118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:06.305786 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "c0f50ae513f4445c8ddafca37e23aca7"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "cb72c51151d6ffafeb94674ab1d1c287"
server_key_iv: "ef94a8cbf6a2613340d29a39e9bc4bbc"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.306164 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "c0f50ae513f4445c8ddafca37e23aca7"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "cb72c51151d6ffafeb94674ab1d1c287"
server_key_iv: "ef94a8cbf6a2613340d29a39e9bc4bbc"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.307654 6118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:06.308298 6133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.308432 6118 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:06.308511 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "c0f50ae513f4445c8ddafca37e23aca7"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "cb72c51151d6ffafeb94674ab1d1c287"
server_key_iv: "ef94a8cbf6a2613340d29a39e9bc4bbc"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.308570 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:06.325590 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:06.327646 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:06.327781 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:06.328261 6118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:06.328310 6118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.328346 6118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:06.328366 6118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:06.335067 6118 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:37871
I20260927 07:09:06.335201 6248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:37871 every 8 connection(s)
I20260927 07:09:06.335457 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:06.337596 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 6118
I20260927 07:09:06.337674 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:06.337849 4793 external_mini_cluster.cc:1428] Setting key e158ef3b7bfcd585c1be4d609bfbe8ad
I20260927 07:09:06.338105 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.328760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:33091 (local address 127.4.174.126:33817)
0927 07:09:06.328825 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.328827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.329086 (+ 259us) server_negotiation.cc:407] Connection header received
0927 07:09:06.329616 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.329618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.329630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.329686 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.330655 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.331200 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.332702 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.332980 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.333038 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.335572 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.335574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.335574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.335582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.336816 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.337180 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.337180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.337181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.337231 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.337441 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.337441 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.337441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.337545 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.337591 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.337933 (+ 342us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.338014 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:06.338253 6142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.328733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33817 (local address 127.4.174.66:33091)
0927 07:09:06.328983 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.329013 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.329512 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.329821 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.329826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.330192 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:06.330493 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.330499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.331286 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.331287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.332589 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.332592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.333057 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.333058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.333218 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.333697 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:06.333706 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:06.335493 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:06.336898 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.336903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.336906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.337118 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.337297 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.337297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.337298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.337395 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.337599 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:06.337605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:06.337765 (+ 160us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.337881 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.338044 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":86,"threads_started":1}
I20260927 07:09:06.338675 6249 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33817
I20260927 07:09:06.338749 6249 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:06.338933 6249 heartbeater.cc:511] Master 127.4.174.126:33817 requested a full tablet report, sending...
I20260927 07:09:06.339313 5905 ts_manager.cc:194] Registered new tserver with Master: c0f50ae513f4445c8ddafca37e23aca7 (127.4.174.66:37871)
I20260927 07:09:06.339767 5905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:33091
I20260927 07:09:06.345016 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.339731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:54735 (local address 127.4.174.126:33817)
0927 07:09:06.339767 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.339769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.339971 (+ 202us) server_negotiation.cc:407] Connection header received
0927 07:09:06.340029 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.340030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.340040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.340082 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.340428 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.340895 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.341766 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.341938 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.341982 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.342878 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.342879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.342879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.342884 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.344044 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.344379 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.344380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.344380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.344422 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.344573 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.344573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.344573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.344707 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.344754 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.344907 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.344957 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 07:09:06.345130 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.339678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33817 (local address 127.4.174.66:54735)
0927 07:09:06.339912 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.339922 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.339982 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.340107 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.340107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.340288 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:06.340357 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.340359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.340982 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.340983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.341698 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.341700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.341996 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.341996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.342065 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.342424 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:06.342429 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:06.342805 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:06.344115 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.344116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.344117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.344315 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.344461 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.344461 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.344462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.344522 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.344764 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:06.344765 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:06.344819 (+ 54us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.344890 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.344982 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:06.363481 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33817
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:06.442322 6258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:06.442517 6258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:06.442538 6258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:06.443712 6258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:06.443753 6258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:06.443773 6258 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:06.443791 6258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:06.445194 6258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:33817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:06.445425 6258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:06.445628 6258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:06.450295 6263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:06.450302 6266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:06.450356 6264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:06.450490 6258 server_base.cc:1034] running on GCE node
I20260927 07:09:06.450745 6258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:06.450978 6258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:06.452095 6258 hybrid_clock.cc:648] HybridClock initialized: now 1790492946452082 us; error 32 us; skew 500 ppm
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:06.454418 6258 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:06.454876 6258 webserver.cc:460] Webserver started at http://127.4.174.67:44935/ using document root <none> and password file <none>
I20260927 07:09:06.455084 6258 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:06.455127 6258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:06.455231 6258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:06.456239 6258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "cf4fd0402035484a914c003dbd9b01fb"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "817464841732a0d52eb3672f32e15b32"
server_key_iv: "a7d02dc34e70f7b110a05361c5728103"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.456580 6258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "cf4fd0402035484a914c003dbd9b01fb"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "817464841732a0d52eb3672f32e15b32"
server_key_iv: "a7d02dc34e70f7b110a05361c5728103"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.457814 6258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:06.458542 6273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.458748 6258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:06.458839 6258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "cf4fd0402035484a914c003dbd9b01fb"
format_stamp: "Formatted at 2026-09-27 07:09:06 on dist-test-slave-zw7p"
server_key: "817464841732a0d52eb3672f32e15b32"
server_key_iv: "a7d02dc34e70f7b110a05361c5728103"
server_key_version: "encryptionkey@0"
I20260927 07:09:06.458910 6258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:06.474927 6258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:06.476749 6258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:06.476881 6258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:06.477458 6258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:06.477515 6258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.477546 6258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:06.477562 6258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:06.484375 6258 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:37741
I20260927 07:09:06.484449 6388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:37741 every 8 connection(s)
I20260927 07:09:06.484777 6258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:06.486413 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.477923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:53549 (local address 127.4.174.126:33817)
0927 07:09:06.477985 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.477987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.478213 (+ 226us) server_negotiation.cc:407] Connection header received
0927 07:09:06.478896 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.478896 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.478907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.478949 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.479787 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.480534 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.481515 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.481807 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.481872 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.483989 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.483990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.483990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.483998 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.485409 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.485755 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.485756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.485756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.485804 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.485959 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.485960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.485960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.486050 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.486093 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.486298 (+ 205us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.486353 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:06.486606 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.477910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33817 (local address 127.4.174.67:53549)
0927 07:09:06.478077 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.478124 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.478789 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.479034 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.479039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.479395 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:06.479697 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.479703 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.480630 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.480631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.481436 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.481438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.481900 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.481901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.482062 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.482598 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:06.482608 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:06.483904 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:06.485473 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.485476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.485477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.485698 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.485839 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.485840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.485840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.485922 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.486092 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:06.486095 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:06.486205 (+ 110us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.486277 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.486388 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260927 07:09:06.487051 6389 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33817
I20260927 07:09:06.487128 6389 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:06.487309 6389 heartbeater.cc:511] Master 127.4.174.126:33817 requested a full tablet report, sending...
I20260927 07:09:06.487663 5906 ts_manager.cc:194] Registered new tserver with Master: cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741)
I20260927 07:09:06.488019 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 6258
I20260927 07:09:06.488101 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:06.488131 5906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:53549
I20260927 07:09:06.488301 4793 external_mini_cluster.cc:1428] Setting key ab5e4eae3d188aff04994d0518cb7118
I20260927 07:09:06.489167 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:09:06.493717 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.488562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33817 (local address 127.4.174.67:50043)
0927 07:09:06.488733 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.488744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.488813 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.488929 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.488929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.489091 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:06.489146 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.489148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.489812 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.489813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.490519 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.490521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.490868 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.490868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.490952 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.491326 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:06.491331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:06.491585 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:06.492839 (+ 1254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.492840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.492841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.492989 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.493157 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.493158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.493158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.493215 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.493413 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:06.493414 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:06.493458 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.493520 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.493618 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260927 07:09:06.493716 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.488604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:50043 (local address 127.4.174.126:33817)
0927 07:09:06.488648 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.488650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.488790 (+ 140us) server_negotiation.cc:407] Connection header received
0927 07:09:06.488873 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.488874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.488883 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.488925 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.489261 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.489718 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.490623 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.490804 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.490858 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.491661 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.491662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.491662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.491669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.492756 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.493050 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.493051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.493052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.493103 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.493263 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.493264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.493264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.493362 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.493415 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.493553 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.493615 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:06.511119 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.504458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46396 (local address 127.4.174.126:33817)
0927 07:09:06.504517 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.504519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.504527 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:06.504605 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.504606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.504617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.504683 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.505166 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.505642 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.506369 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.506676 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.506744 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.508830 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.508831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.508832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.508840 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.509964 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.510289 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.510290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.510290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.510348 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.510524 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.510525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.510525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.510673 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.510770 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.510899 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.510974 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 07:09:06.512681 5906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:06.514217 5906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:06.519929 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.516345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:46779 (local address 127.0.0.1:50510)
0927 07:09:06.516543 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.516552 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.516594 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.516770 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.516773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.516775 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:06.516855 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.516857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.517754 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.517754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.518814 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.518816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.519634 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.519635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.519727 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.519741 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.519811 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.519857 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":37,"threads_started":1}
I20260927 07:09:06.519966 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.516449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.4.174.65:46779)
0927 07:09:06.516598 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.516603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.516625 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:09:06.516664 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.516666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.516681 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.516759 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:06.516935 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.517679 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.518887 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.519548 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.519618 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.519674 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.519833 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.519871 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260927 07:09:06.520581 6405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.516454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:37871 (local address 127.0.0.1:58330)
0927 07:09:06.516651 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.516659 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.516690 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.516959 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.516960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.516961 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:06.517017 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.517017 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.518254 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.518255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.519190 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.519192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.520355 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.520356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.520416 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.520421 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.520474 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.520512 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":3,"thread_start_us":43,"threads_started":1}
I20260927 07:09:06.520625 6043 tablet_service.cc:1467] Processing CreateTablet for tablet d0accf55dc604ea088a8eeac9e260f03 (DEFAULT_TABLE table=test-table [id=c91be00142fb4a86bb0258e132846f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:06.520615 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.516568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.4.174.66:37871)
0927 07:09:06.516734 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.516739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.516748 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:06.516796 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.516799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.516815 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.516890 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:06.517086 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.518181 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.519258 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.519938 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.520342 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.520407 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.520488 (+ 81us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.520523 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260927 07:09:06.521013 6043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0accf55dc604ea088a8eeac9e260f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.521399 6403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.516153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37741 (local address 127.0.0.1:59874)
0927 07:09:06.516265 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.516303 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.516372 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.516965 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.516965 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.516967 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:06.517016 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.517017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.519254 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.519254 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.520056 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.520058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.521186 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.521187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.521247 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.521251 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.521302 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.521337 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260927 07:09:06.521430 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.516259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.4.174.67:37741)
0927 07:09:06.516788 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.516791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.516799 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:06.516830 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.516832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.516846 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.516906 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:06.517086 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.518172 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.520115 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.520958 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.521095 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.521156 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.521321 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.521356 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":89,"threads_started":1}
I20260927 07:09:06.521986 6321 tablet_service.cc:1467] Processing CreateTablet for tablet d0accf55dc604ea088a8eeac9e260f03 (DEFAULT_TABLE table=test-table [id=c91be00142fb4a86bb0258e132846f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:06.522260 6321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0accf55dc604ea088a8eeac9e260f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.523128 6183 tablet_service.cc:1467] Processing CreateTablet for tablet d0accf55dc604ea088a8eeac9e260f03 (DEFAULT_TABLE table=test-table [id=c91be00142fb4a86bb0258e132846f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:06.523444 6183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0accf55dc604ea088a8eeac9e260f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.524156 6409 tablet_bootstrap.cc:492] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Bootstrap starting.
I20260927 07:09:06.524680 6409 tablet_bootstrap.cc:654] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.524950 6409 log.cc:824] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:06.525180 6410 tablet_bootstrap.cc:492] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Bootstrap starting.
I20260927 07:09:06.526010 6410 tablet_bootstrap.cc:654] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.526268 6410 log.cc:824] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:06.526582 6413 tablet_bootstrap.cc:492] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Bootstrap starting.
I20260927 07:09:06.526993 6409 tablet_bootstrap.cc:492] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: No bootstrap required, opened a new log
I20260927 07:09:06.527010 6410 tablet_bootstrap.cc:492] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: No bootstrap required, opened a new log
I20260927 07:09:06.527045 6409 ts_tablet_manager.cc:1397] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 07:09:06.527064 6410 ts_tablet_manager.cc:1397] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:06.527346 6413 tablet_bootstrap.cc:654] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.527696 6413 log.cc:824] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:06.528492 6413 tablet_bootstrap.cc:492] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: No bootstrap required, opened a new log
I20260927 07:09:06.528448 6409 raft_consensus.cc:348] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.528568 6409 raft_consensus.cc:374] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.528590 6409 raft_consensus.cc:729] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a3fd3d811a4418b3096c572445f1a4, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.528738 6409 consensus_queue.cc:260] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.528784 6413 ts_tablet_manager.cc:1397] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:06.528860 6410 raft_consensus.cc:348] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.528997 6409 ts_tablet_manager.cc:1428] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:06.529008 6410 raft_consensus.cc:374] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.529043 6410 raft_consensus.cc:729] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf4fd0402035484a914c003dbd9b01fb, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.529127 6109 heartbeater.cc:503] Master 127.4.174.126:33817 was elected leader, sending a full tablet report...
I20260927 07:09:06.529170 6410 consensus_queue.cc:260] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.529414 6410 ts_tablet_manager.cc:1428] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 07:09:06.529523 6389 heartbeater.cc:503] Master 127.4.174.126:33817 was elected leader, sending a full tablet report...
I20260927 07:09:06.530937 6413 raft_consensus.cc:348] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.531057 6413 raft_consensus.cc:374] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.531077 6413 raft_consensus.cc:729] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f50ae513f4445c8ddafca37e23aca7, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.531163 6413 consensus_queue.cc:260] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.531391 6413 ts_tablet_manager.cc:1428] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:06.531447 6249 heartbeater.cc:503] Master 127.4.174.126:33817 was elected leader, sending a full tablet report...
W20260927 07:09:06.585994 6250 tablet.cc:2369] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:09:06.685321 6110 tablet.cc:2369] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:06.706799 6417 raft_consensus.cc:482] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:06.706919 6417 raft_consensus.cc:504] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.707244 6417 leader_election.cc:288] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741), 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779)
I20260927 07:09:06.709897 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.707494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:42279 (local address 127.4.174.67:37741)
0927 07:09:06.707542 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.707544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.707554 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:06.707604 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.707605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.707615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.707715 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:06.707861 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.708358 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.709239 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.709594 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.709656 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.709666 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.709789 (+ 123us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.709825 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:09:06.709939 6142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.707437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37741 (local address 127.4.174.66:42279)
0927 07:09:06.707487 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.707499 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.707545 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.707727 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.707728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.707730 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:06.707789 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.707790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.708426 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.708427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.709156 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.709158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.709660 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.709661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.709733 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.709736 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.709779 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.709823 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":5}
I20260927 07:09:06.709950 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.707629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:52109 (local address 127.4.174.65:46779)
0927 07:09:06.707683 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.707685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.707784 (+ 99us) server_negotiation.cc:407] Connection header received
0927 07:09:06.707829 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.707830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.707837 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.707882 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:06.708013 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.708469 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.709310 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.709656 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.709715 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.709722 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.709849 (+ 127us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.709885 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 07:09:06.710026 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.707556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:46779 (local address 127.4.174.66:52109)
0927 07:09:06.707714 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.707725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.707779 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.707888 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.707889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.707890 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:06.707960 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.707961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.708534 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.708535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.709238 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.709240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.709720 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.709721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.709792 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.709795 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.709839 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.709875 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260927 07:09:06.710274 6063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0accf55dc604ea088a8eeac9e260f03" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a3fd3d811a4418b3096c572445f1a4" is_pre_election: true
I20260927 07:09:06.710285 6343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0accf55dc604ea088a8eeac9e260f03" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf4fd0402035484a914c003dbd9b01fb" is_pre_election: true
I20260927 07:09:06.710423 6343 raft_consensus.cc:2454] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 0.
I20260927 07:09:06.710423 6063 raft_consensus.cc:2454] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 0.
I20260927 07:09:06.710670 6136 leader_election.cc:302] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81a3fd3d811a4418b3096c572445f1a4, c0f50ae513f4445c8ddafca37e23aca7; no voters:
I20260927 07:09:06.710825 6417 raft_consensus.cc:2781] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:06.710880 6417 raft_consensus.cc:482] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:06.710896 6417 raft_consensus.cc:3037] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:06.711716 6417 raft_consensus.cc:504] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.711826 6417 leader_election.cc:288] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 election: Requested vote from peers cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741), 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779)
I20260927 07:09:06.711990 6343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0accf55dc604ea088a8eeac9e260f03" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf4fd0402035484a914c003dbd9b01fb"
I20260927 07:09:06.712030 6063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0accf55dc604ea088a8eeac9e260f03" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a3fd3d811a4418b3096c572445f1a4"
I20260927 07:09:06.712069 6343 raft_consensus.cc:3037] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:06.712109 6063 raft_consensus.cc:3037] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:06.712716 6343 raft_consensus.cc:2454] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 1.
I20260927 07:09:06.712808 6063 raft_consensus.cc:2454] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 1.
I20260927 07:09:06.712864 6136 leader_election.cc:302] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f50ae513f4445c8ddafca37e23aca7, cf4fd0402035484a914c003dbd9b01fb; no voters:
I20260927 07:09:06.712963 6417 raft_consensus.cc:2781] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:06.713088 6417 raft_consensus.cc:686] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 LEADER]: Becoming Leader. State: Replica: c0f50ae513f4445c8ddafca37e23aca7, State: Running, Role: LEADER
I20260927 07:09:06.713235 6417 consensus_queue.cc:237] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } }
I20260927 07:09:06.714061 5906 catalog_manager.cc:5495] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0f50ae513f4445c8ddafca37e23aca7 (127.4.174.66). New cstate: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } health_report { overall_health: HEALTHY } } }
W20260927 07:09:06.735312 6390 tablet.cc:2369] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:06.746867 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.744527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.4.174.66:37871)
0927 07:09:06.744586 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.744588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.744594 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:09:06.744626 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.744626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.744642 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.744695 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:06.744836 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.745303 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.746060 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.746245 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.746299 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.746492 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:06.746616 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:06.746685 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.746729 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.746756 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 07:09:06.748520 6063 raft_consensus.cc:1261] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Refusing update from remote peer c0f50ae513f4445c8ddafca37e23aca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:06.748558 6343 raft_consensus.cc:1261] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Refusing update from remote peer c0f50ae513f4445c8ddafca37e23aca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:06.748821 6417 consensus_queue.cc:1035] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:06.748945 6419 consensus_queue.cc:1035] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:06.751077 6422 mvcc.cc:205] Tried to move back new op lower bound from 7333859109879611392 to 7333859109737951232. Current Snapshot: MvccSnapshot[applied={T|T < 7333859109879611392}]
I20260927 07:09:06.751791 6423 mvcc.cc:205] Tried to move back new op lower bound from 7333859109879611392 to 7333859109737951232. Current Snapshot: MvccSnapshot[applied={T|T < 7333859109879611392}]
I20260927 07:09:06.751822 6424 mvcc.cc:205] Tried to move back new op lower bound from 7333859109879611392 to 7333859109737951232. Current Snapshot: MvccSnapshot[applied={T|T < 7333859109879611392}]
I20260927 07:09:06.754215 5906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:06.754295 5906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:06.754976 5906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266: Sending DeleteTablet for 3 replicas of tablet d0accf55dc604ea088a8eeac9e260f03
I20260927 07:09:06.755261 6321 tablet_service.cc:1514] Processing DeleteTablet for tablet d0accf55dc604ea088a8eeac9e260f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:06 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:59874
I20260927 07:09:06.755291 6043 tablet_service.cc:1514] Processing DeleteTablet for tablet d0accf55dc604ea088a8eeac9e260f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:06 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50510
I20260927 07:09:06.755458 6431 tablet_replica.cc:330] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: stopping tablet replica
I20260927 07:09:06.755575 6431 raft_consensus.cc:2229] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:06.755657 6183 tablet_service.cc:1514] Processing DeleteTablet for tablet d0accf55dc604ea088a8eeac9e260f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:06 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:58330
I20260927 07:09:06.755795 6431 raft_consensus.cc:2258] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:06.755736 5906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:06.755877 5906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:06.756044 6432 tablet_replica.cc:330] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: stopping tablet replica
I20260927 07:09:06.756138 6432 raft_consensus.cc:2229] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:06.756239 6432 raft_consensus.cc:2258] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:06.756389 6433 tablet_replica.cc:330] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: stopping tablet replica
I20260927 07:09:06.756475 6433 raft_consensus.cc:2229] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:06.756560 6433 raft_consensus.cc:2258] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:06.756892 6432 ts_tablet_manager.cc:1905] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:06.757107 6321 tablet_service.cc:1467] Processing CreateTablet for tablet 6afdfac1682e476ca56c6b70ed94ee04 (DEFAULT_TABLE table=test-table [id=af8b9ea05d714d2a8fbd3ed4c9406b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:06.757195 6321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6afdfac1682e476ca56c6b70ed94ee04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.757242 6183 tablet_service.cc:1467] Processing CreateTablet for tablet 6afdfac1682e476ca56c6b70ed94ee04 (DEFAULT_TABLE table=test-table [id=af8b9ea05d714d2a8fbd3ed4c9406b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:06.757355 6183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6afdfac1682e476ca56c6b70ed94ee04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.758126 6410 tablet_bootstrap.cc:492] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: Bootstrap starting.
I20260927 07:09:06.758529 6410 tablet_bootstrap.cc:654] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.758646 6413 tablet_bootstrap.cc:492] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: Bootstrap starting.
I20260927 07:09:06.759032 6413 tablet_bootstrap.cc:654] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.759392 6043 tablet_service.cc:1467] Processing CreateTablet for tablet 6afdfac1682e476ca56c6b70ed94ee04 (DEFAULT_TABLE table=test-table [id=af8b9ea05d714d2a8fbd3ed4c9406b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:06.759531 6043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6afdfac1682e476ca56c6b70ed94ee04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.759615 6433 ts_tablet_manager.cc:1905] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:06.760658 6432 ts_tablet_manager.cc:1918] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:06.760740 6432 log.cc:1196] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/d0accf55dc604ea088a8eeac9e260f03
I20260927 07:09:06.760845 6409 tablet_bootstrap.cc:492] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: Bootstrap starting.
I20260927 07:09:06.760977 6432 ts_tablet_manager.cc:1939] T d0accf55dc604ea088a8eeac9e260f03 P cf4fd0402035484a914c003dbd9b01fb: Deleting consensus metadata
I20260927 07:09:06.761202 6409 tablet_bootstrap.cc:654] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.761247 6413 tablet_bootstrap.cc:492] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: No bootstrap required, opened a new log
I20260927 07:09:06.761298 6413 ts_tablet_manager.cc:1397] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 07:09:06.761371 6433 ts_tablet_manager.cc:1918] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:06.761438 6433 log.cc:1196] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/d0accf55dc604ea088a8eeac9e260f03
I20260927 07:09:06.761435 6413 raft_consensus.cc:348] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.761518 6413 raft_consensus.cc:374] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.761554 6413 raft_consensus.cc:729] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f50ae513f4445c8ddafca37e23aca7, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.761595 6413 consensus_queue.cc:260] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.761685 6413 ts_tablet_manager.cc:1428] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.761696 6433 ts_tablet_manager.cc:1939] T d0accf55dc604ea088a8eeac9e260f03 P c0f50ae513f4445c8ddafca37e23aca7: Deleting consensus metadata
I20260927 07:09:06.761847 5892 catalog_manager.cc:4841] TS cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741): tablet d0accf55dc604ea088a8eeac9e260f03 (table test-table [id=c91be00142fb4a86bb0258e132846f2e]) successfully deleted
I20260927 07:09:06.762073 5892 catalog_manager.cc:4841] TS c0f50ae513f4445c8ddafca37e23aca7 (127.4.174.66:37871): tablet d0accf55dc604ea088a8eeac9e260f03 (table test-table [id=c91be00142fb4a86bb0258e132846f2e]) successfully deleted
I20260927 07:09:06.762722 6431 ts_tablet_manager.cc:1905] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:06.764077 6431 ts_tablet_manager.cc:1918] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:06.764156 6431 log.cc:1196] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/d0accf55dc604ea088a8eeac9e260f03
I20260927 07:09:06.764376 6431 ts_tablet_manager.cc:1939] T d0accf55dc604ea088a8eeac9e260f03 P 81a3fd3d811a4418b3096c572445f1a4: Deleting consensus metadata
I20260927 07:09:06.764755 5892 catalog_manager.cc:4841] TS 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779): tablet d0accf55dc604ea088a8eeac9e260f03 (table test-table [id=c91be00142fb4a86bb0258e132846f2e]) successfully deleted
I20260927 07:09:06.764837 6410 tablet_bootstrap.cc:492] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: No bootstrap required, opened a new log
I20260927 07:09:06.764883 6410 ts_tablet_manager.cc:1397] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260927 07:09:06.765017 6410 raft_consensus.cc:348] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.765070 6410 raft_consensus.cc:374] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.765087 6410 raft_consensus.cc:729] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf4fd0402035484a914c003dbd9b01fb, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.765127 6410 consensus_queue.cc:260] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.765216 6410 ts_tablet_manager.cc:1428] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.766873 6409 tablet_bootstrap.cc:492] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: No bootstrap required, opened a new log
I20260927 07:09:06.766914 6409 ts_tablet_manager.cc:1397] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260927 07:09:06.767171 6409 raft_consensus.cc:348] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.767225 6409 raft_consensus.cc:374] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.767243 6409 raft_consensus.cc:729] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a3fd3d811a4418b3096c572445f1a4, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.767283 6409 consensus_queue.cc:260] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.767386 6409 ts_tablet_manager.cc:1428] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:06.877846 6419 raft_consensus.cc:482] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:06.877940 6419 raft_consensus.cc:504] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.878072 6419 leader_election.cc:288] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779), cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741)
I20260927 07:09:06.878265 6063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afdfac1682e476ca56c6b70ed94ee04" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a3fd3d811a4418b3096c572445f1a4" is_pre_election: true
I20260927 07:09:06.878360 6063 raft_consensus.cc:2454] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 0.
I20260927 07:09:06.878314 6343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afdfac1682e476ca56c6b70ed94ee04" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf4fd0402035484a914c003dbd9b01fb" is_pre_election: true
I20260927 07:09:06.878408 6343 raft_consensus.cc:2454] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 0.
I20260927 07:09:06.878508 6136 leader_election.cc:302] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81a3fd3d811a4418b3096c572445f1a4, c0f50ae513f4445c8ddafca37e23aca7; no voters:
I20260927 07:09:06.878639 6419 raft_consensus.cc:2781] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:06.878681 6419 raft_consensus.cc:482] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:06.878700 6419 raft_consensus.cc:3037] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:06.879390 6419 raft_consensus.cc:504] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.879499 6419 leader_election.cc:288] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 election: Requested vote from peers 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779), cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741)
I20260927 07:09:06.879670 6063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afdfac1682e476ca56c6b70ed94ee04" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a3fd3d811a4418b3096c572445f1a4"
I20260927 07:09:06.879674 6343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afdfac1682e476ca56c6b70ed94ee04" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf4fd0402035484a914c003dbd9b01fb"
I20260927 07:09:06.879738 6063 raft_consensus.cc:3037] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:06.879739 6343 raft_consensus.cc:3037] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:06.880460 6063 raft_consensus.cc:2454] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 1.
I20260927 07:09:06.880460 6343 raft_consensus.cc:2454] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 1.
I20260927 07:09:06.880615 6136 leader_election.cc:302] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f50ae513f4445c8ddafca37e23aca7, cf4fd0402035484a914c003dbd9b01fb; no voters:
I20260927 07:09:06.880699 6419 raft_consensus.cc:2781] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:06.880756 6419 raft_consensus.cc:686] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 LEADER]: Becoming Leader. State: Replica: c0f50ae513f4445c8ddafca37e23aca7, State: Running, Role: LEADER
I20260927 07:09:06.880803 6419 consensus_queue.cc:237] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.881340 5905 catalog_manager.cc:5495] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0f50ae513f4445c8ddafca37e23aca7 (127.4.174.66). New cstate: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } health_report { overall_health: UNKNOWN } } }
Sep 27 07:09:06 dist-test-slave-zw7p krb5kdc[5861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492945, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:06.937312 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.931209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33817 (local address 127.0.0.1:46404)
0927 07:09:06.931358 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:06.931370 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:06.931443 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:06.931585 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:06.931587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:06.931897 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:06.931945 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.931948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.932555 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.932556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.933113 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:06.933115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.933420 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.933421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:06.933469 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.933915 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:06.933920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:06.935223 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:06.936482 (+ 1259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.936488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.936491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.936694 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.936837 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:06.936837 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:06.936837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:06.936886 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:06.937047 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:06.937051 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:06.937086 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:09:06.937135 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 07:09:06.937215 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260927 07:09:06.937352 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:06.931305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.4.174.126:33817)
0927 07:09:06.931378 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:06.931379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:06.931431 (+ 52us) server_negotiation.cc:407] Connection header received
0927 07:09:06.931485 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:06.931486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:06.931495 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:06.931545 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:06.932012 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.932491 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.933171 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:06.933363 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:06.933407 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:06.935292 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:06.935293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:06.935293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:06.935299 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:06.936417 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.936751 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.936752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.936752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.936796 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:06.936922 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:06.936922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:06.936922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:06.937010 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:06.937054 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:06.937154 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 07:09:06.937253 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":47}
I20260927 07:09:06.939116 5905 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:46404:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 07:09:06.939296 5905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:06.940714 6183 tablet_service.cc:1467] Processing CreateTablet for tablet afae4bf29ca14c3d95b455a28c9b860e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0e2baafa5964ac4bc9f3caed0b47b50]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:06.940750 6043 tablet_service.cc:1467] Processing CreateTablet for tablet afae4bf29ca14c3d95b455a28c9b860e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0e2baafa5964ac4bc9f3caed0b47b50]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:06.940773 6321 tablet_service.cc:1467] Processing CreateTablet for tablet afae4bf29ca14c3d95b455a28c9b860e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0e2baafa5964ac4bc9f3caed0b47b50]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:06.940883 6183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afae4bf29ca14c3d95b455a28c9b860e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.940888 6321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afae4bf29ca14c3d95b455a28c9b860e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.940896 6043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afae4bf29ca14c3d95b455a28c9b860e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:06.942040 6409 tablet_bootstrap.cc:492] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: Bootstrap starting.
I20260927 07:09:06.942308 6413 tablet_bootstrap.cc:492] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: Bootstrap starting.
I20260927 07:09:06.942471 6409 tablet_bootstrap.cc:654] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.942651 6413 tablet_bootstrap.cc:654] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.942905 6410 tablet_bootstrap.cc:492] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: Bootstrap starting.
I20260927 07:09:06.943167 6409 tablet_bootstrap.cc:492] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: No bootstrap required, opened a new log
I20260927 07:09:06.943226 6409 ts_tablet_manager.cc:1397] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:06.943332 6410 tablet_bootstrap.cc:654] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:06.943389 6409 raft_consensus.cc:348] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.943480 6409 raft_consensus.cc:374] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.943512 6409 raft_consensus.cc:729] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a3fd3d811a4418b3096c572445f1a4, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.943578 6409 consensus_queue.cc:260] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.943691 6409 ts_tablet_manager.cc:1428] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:06.943861 6415 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:06.943929 6410 tablet_bootstrap.cc:492] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: No bootstrap required, opened a new log
I20260927 07:09:06.943951 6415 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:06.943969 6410 ts_tablet_manager.cc:1397] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:06.944109 6413 tablet_bootstrap.cc:492] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: No bootstrap required, opened a new log
I20260927 07:09:06.944146 6413 ts_tablet_manager.cc:1397] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 07:09:06.944105 6410 raft_consensus.cc:348] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.944192 6410 raft_consensus.cc:374] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.944211 6410 raft_consensus.cc:729] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf4fd0402035484a914c003dbd9b01fb, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.944249 6410 consensus_queue.cc:260] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.944242 6413 raft_consensus.cc:348] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.944296 6413 raft_consensus.cc:374] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:06.944315 6413 raft_consensus.cc:729] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f50ae513f4445c8ddafca37e23aca7, State: Initialized, Role: FOLLOWER
I20260927 07:09:06.944329 6410 ts_tablet_manager.cc:1428] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.944355 6413 consensus_queue.cc:260] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.944438 6413 ts_tablet_manager.cc:1428] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:06.944392 6416 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:06.944496 6416 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:06.944465 6419 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:06.944546 6419 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:06.998667 6419 raft_consensus.cc:482] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:06.998767 6419 raft_consensus.cc:504] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:06.998914 6419 leader_election.cc:288] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779), cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741)
I20260927 07:09:06.999085 6063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afae4bf29ca14c3d95b455a28c9b860e" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a3fd3d811a4418b3096c572445f1a4" is_pre_election: true
I20260927 07:09:06.999198 6063 raft_consensus.cc:2454] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 0.
I20260927 07:09:06.999140 6343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afae4bf29ca14c3d95b455a28c9b860e" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf4fd0402035484a914c003dbd9b01fb" is_pre_election: true
I20260927 07:09:06.999244 6343 raft_consensus.cc:2454] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 0.
I20260927 07:09:06.999377 6136 leader_election.cc:302] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81a3fd3d811a4418b3096c572445f1a4, c0f50ae513f4445c8ddafca37e23aca7; no voters:
I20260927 07:09:06.999477 6419 raft_consensus.cc:2781] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:06.999528 6419 raft_consensus.cc:482] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:06.999553 6419 raft_consensus.cc:3037] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:07.000218 6419 raft_consensus.cc:504] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:07.000357 6419 leader_election.cc:288] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 election: Requested vote from peers 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779), cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741)
I20260927 07:09:07.000560 6063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afae4bf29ca14c3d95b455a28c9b860e" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a3fd3d811a4418b3096c572445f1a4"
I20260927 07:09:07.000615 6343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afae4bf29ca14c3d95b455a28c9b860e" candidate_uuid: "c0f50ae513f4445c8ddafca37e23aca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf4fd0402035484a914c003dbd9b01fb"
I20260927 07:09:07.000689 6063 raft_consensus.cc:3037] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:07.000690 6343 raft_consensus.cc:3037] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:07.001420 6343 raft_consensus.cc:2454] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 1.
I20260927 07:09:07.001420 6063 raft_consensus.cc:2454] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f50ae513f4445c8ddafca37e23aca7 in term 1.
I20260927 07:09:07.001595 6136 leader_election.cc:302] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81a3fd3d811a4418b3096c572445f1a4, c0f50ae513f4445c8ddafca37e23aca7; no voters:
I20260927 07:09:07.001678 6419 raft_consensus.cc:2781] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:07.001739 6419 raft_consensus.cc:686] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [term 1 LEADER]: Becoming Leader. State: Replica: c0f50ae513f4445c8ddafca37e23aca7, State: Running, Role: LEADER
I20260927 07:09:07.001787 6419 consensus_queue.cc:237] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } }
I20260927 07:09:07.002039 6417 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: TxnStatusTablet state changed. Reason: New leader c0f50ae513f4445c8ddafca37e23aca7. Latest consensus state: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:07.002173 6417 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:07.002378 5904 catalog_manager.cc:5495] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0f50ae513f4445c8ddafca37e23aca7 (127.4.174.66). New cstate: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:07.002707 6448 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 07:09:07.002842 6448 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 07:09:07.018249 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:07.015802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:37871 (local address 127.0.0.1:58354)
0927 07:09:07.015868 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:07.015880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:07.015928 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:07.016054 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:07.016055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:07.016056 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:07.016098 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:07.016100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.016940 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.016941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:07.017490 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:07.017492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.017927 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.017928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:07.017974 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:07.017988 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:07.018126 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:07.018130 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:07.018158 (+ 28us) client_negotiation.cc:240] Negotiation successful
0927 07:09:07.018192 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 07:09:07.018260 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:07.015868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.4.174.66:37871)
0927 07:09:07.015914 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:07.015916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:07.015926 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:07.015981 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:07.015981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:07.015993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:07.016049 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:07.016132 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.016881 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.017553 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.017857 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.017915 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:07.018048 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:07.018081 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:07.018125 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:07.018163 (+ 38us) server_negotiation.cc:299] Negotiation successful
0927 07:09:07.018183 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:07.019294 6343 raft_consensus.cc:1261] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Refusing update from remote peer c0f50ae513f4445c8ddafca37e23aca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:07.019302 6063 raft_consensus.cc:1261] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Refusing update from remote peer c0f50ae513f4445c8ddafca37e23aca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:07.019532 6417 consensus_queue.cc:1035] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:07.019658 6419 consensus_queue.cc:1035] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:07.020041 6415 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: TxnStatusTablet state changed. Reason: New leader c0f50ae513f4445c8ddafca37e23aca7. Latest consensus state: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:07.020130 6415 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:07.020244 6416 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: TxnStatusTablet state changed. Reason: New leader c0f50ae513f4445c8ddafca37e23aca7. Latest consensus state: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:07.020352 6416 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:07.021052 6419 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:07.021121 6419 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:07.021158 6419 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:07.021190 6419 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P c0f50ae513f4445c8ddafca37e23aca7: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:07.021642 6415 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:07.021724 6415 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P 81a3fd3d811a4418b3096c572445f1a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:07.022015 6416 tablet_replica.cc:439] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0f50ae513f4445c8ddafca37e23aca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 } } peers { permanent_uuid: "c0f50ae513f4445c8ddafca37e23aca7" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 37871 } } peers { permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 } } }
I20260927 07:09:07.022109 6416 tablet_replica.cc:442] T afae4bf29ca14c3d95b455a28c9b860e P cf4fd0402035484a914c003dbd9b01fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:07.027101 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:07.024632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:37871 (local address 127.4.174.66:45915)
0927 07:09:07.024797 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:07.024806 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:07.024854 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:07.024955 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:07.024956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:07.024957 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:07.024992 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:07.024993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.025789 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.025790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:07.026333 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:07.026335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.026773 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.026774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:07.026808 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:07.026820 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:07.026960 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:07.026961 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 07:09:07.026987 (+ 26us) client_negotiation.cc:240] Negotiation successful
0927 07:09:07.027017 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260927 07:09:07.027105 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:07.024690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:45915 (local address 127.4.174.66:37871)
0927 07:09:07.024739 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:07.024740 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:07.024857 (+ 117us) server_negotiation.cc:407] Connection header received
0927 07:09:07.024883 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:07.024884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:07.024893 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:07.024954 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:07.025027 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.025724 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.026394 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.026715 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.026776 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:07.026883 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:07.026912 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:07.026961 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:07.026992 (+ 31us) server_negotiation.cc:299] Negotiation successful
0927 07:09:07.027009 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 07:09:07.029073 6343 raft_consensus.cc:1261] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Refusing update from remote peer c0f50ae513f4445c8ddafca37e23aca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:07.029081 6063 raft_consensus.cc:1261] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Refusing update from remote peer c0f50ae513f4445c8ddafca37e23aca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:07.029250 6417 consensus_queue.cc:1035] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a3fd3d811a4418b3096c572445f1a4" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:07.029348 6453 consensus_queue.cc:1035] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf4fd0402035484a914c003dbd9b01fb" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:07.045086 5906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:07.045189 5906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:07.045773 5906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32b753d6afad46c8896e3201a7593266: Sending DeleteTablet for 3 replicas of tablet 6afdfac1682e476ca56c6b70ed94ee04
I20260927 07:09:07.046017 6182 tablet_service.cc:1514] Processing DeleteTablet for tablet 6afdfac1682e476ca56c6b70ed94ee04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:07 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:58330
I20260927 07:09:07.046111 6433 tablet_replica.cc:330] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: stopping tablet replica
I20260927 07:09:07.046099 6043 tablet_service.cc:1514] Processing DeleteTablet for tablet 6afdfac1682e476ca56c6b70ed94ee04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:07 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50510
I20260927 07:09:07.046156 6433 raft_consensus.cc:2229] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:07.046187 6431 tablet_replica.cc:330] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: stopping tablet replica
I20260927 07:09:07.046213 6433 raft_consensus.cc:2258] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:07.046231 6431 raft_consensus.cc:2229] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:07.046276 6431 raft_consensus.cc:2258] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:07.046584 6321 tablet_service.cc:1514] Processing DeleteTablet for tablet 6afdfac1682e476ca56c6b70ed94ee04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:07 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:59874
I20260927 07:09:07.046619 6431 ts_tablet_manager.cc:1905] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:07.046703 6432 tablet_replica.cc:330] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: stopping tablet replica
I20260927 07:09:07.046754 6432 raft_consensus.cc:2229] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:07.046790 6432 raft_consensus.cc:2258] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:07.048377 6432 ts_tablet_manager.cc:1905] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:07.048441 6433 ts_tablet_manager.cc:1905] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:07.049683 6432 ts_tablet_manager.cc:1918] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:07.049741 6432 log.cc:1196] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/6afdfac1682e476ca56c6b70ed94ee04
I20260927 07:09:07.049913 6432 ts_tablet_manager.cc:1939] T 6afdfac1682e476ca56c6b70ed94ee04 P cf4fd0402035484a914c003dbd9b01fb: Deleting consensus metadata
I20260927 07:09:07.049909 6433 ts_tablet_manager.cc:1918] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:07.049960 6433 log.cc:1196] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/6afdfac1682e476ca56c6b70ed94ee04
I20260927 07:09:07.050181 6433 ts_tablet_manager.cc:1939] T 6afdfac1682e476ca56c6b70ed94ee04 P c0f50ae513f4445c8ddafca37e23aca7: Deleting consensus metadata
I20260927 07:09:07.050297 5892 catalog_manager.cc:4841] TS cf4fd0402035484a914c003dbd9b01fb (127.4.174.67:37741): tablet 6afdfac1682e476ca56c6b70ed94ee04 (table test-table [id=af8b9ea05d714d2a8fbd3ed4c9406b2c]) successfully deleted
I20260927 07:09:07.050544 6431 ts_tablet_manager.cc:1918] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:07.050602 5892 catalog_manager.cc:4841] TS c0f50ae513f4445c8ddafca37e23aca7 (127.4.174.66:37871): tablet 6afdfac1682e476ca56c6b70ed94ee04 (table test-table [id=af8b9ea05d714d2a8fbd3ed4c9406b2c]) successfully deleted
I20260927 07:09:07.050627 6431 log.cc:1196] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/6afdfac1682e476ca56c6b70ed94ee04
I20260927 07:09:07.050843 6431 ts_tablet_manager.cc:1939] T 6afdfac1682e476ca56c6b70ed94ee04 P 81a3fd3d811a4418b3096c572445f1a4: Deleting consensus metadata
I20260927 07:09:07.051126 5892 catalog_manager.cc:4841] TS 81a3fd3d811a4418b3096c572445f1a4 (127.4.174.65:46779): tablet 6afdfac1682e476ca56c6b70ed94ee04 (table test-table [id=af8b9ea05d714d2a8fbd3ed4c9406b2c]) successfully deleted
Sep 27 07:09:07 dist-test-slave-zw7p krb5kdc[5861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:07.059453 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:07.046925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.4.174.65:46779)
0927 07:09:07.046964 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:07.046966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:07.051195 (+ 4229us) server_negotiation.cc:407] Connection header received
0927 07:09:07.051290 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:07.051291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:07.051303 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:07.051346 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:07.051988 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.052447 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.053444 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.053618 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.053659 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:07.056844 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:07.056856 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:07.056856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:07.056864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:07.058361 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:07.058774 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:07.058778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:07.058778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:07.058833 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:07.059004 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:07.059005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:07.059005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:07.059137 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:07.059235 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:07.059320 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 07:09:07.059383 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 07:09:07.059947 6103 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50520
I20260927 07:09:07.066409 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:07.060901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46406 (local address 127.4.174.126:33817)
0927 07:09:07.060947 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:07.060949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:07.060958 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:07.060985 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:07.060987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:07.060996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:07.061040 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:07.061540 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.062104 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.062862 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:07.063058 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:07.063113 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:07.064159 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:07.064161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:07.064161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:07.064169 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:07.065438 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:07.065727 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:07.065728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:07.065729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:07.065786 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:07.065942 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:07.065943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:07.065944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:07.066055 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:07.066139 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:07.066236 (+ 97us) server_negotiation.cc:299] Negotiation successful
0927 07:09:07.066316 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 07:09:07.066648 5906 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46406
I20260927 07:09:07.067240 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 5978
I20260927 07:09:07.072927 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 6118
I20260927 07:09:07.078457 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 6258
I20260927 07:09:07.082329 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 5874
2026-09-27T07:09:07Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2266 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:07 dist-test-slave-zw7p krb5kdc[6466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:07 dist-test-slave-zw7p krb5kdc[6466](info): set up 2 sockets
Sep 27 07:09:07 dist-test-slave-zw7p krb5kdc[6466](info): commencing operation
krb5kdc: starting...
W20260927 07:09:08.022121 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:08.133517 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:38591
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:38591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:08.212697 6479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:08.212904 6479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:08.212925 6479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:08.214085 6479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:08.214128 6479 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 07:09:08.214139 6479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:08.214151 6479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:08.214162 6479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:08.214172 6479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:08.215677 6479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:38591
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:38591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:08.215903 6479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:08.216125 6479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:08.220935 6484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:08.220988 6479 server_base.cc:1034] running on GCE node
W20260927 07:09:08.220969 6487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:08.221017 6485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:08.221391 6479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:08.221692 6479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:08.222839 6479 hybrid_clock.cc:648] HybridClock initialized: now 1790492948222826 us; error 30 us; skew 500 ppm
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:08.225446 6479 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:08.225907 6479 webserver.cc:460] Webserver started at http://127.4.174.126:34245/ using document root <none> and password file <none>
I20260927 07:09:08.226107 6479 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:08.226155 6479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:08.226276 6479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:08.227355 6479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "ac56a3ba9cf24989b56c9da254b0c6c4"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "edf766243348e6ff8f0cadd1d9c86091"
server_key_iv: "c13a9e7a5672adadd5ed623066ca4849"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.227684 6479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "ac56a3ba9cf24989b56c9da254b0c6c4"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "edf766243348e6ff8f0cadd1d9c86091"
server_key_iv: "c13a9e7a5672adadd5ed623066ca4849"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.228935 6479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:08.229557 6494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.229738 6479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:08.229822 6479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "ac56a3ba9cf24989b56c9da254b0c6c4"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "edf766243348e6ff8f0cadd1d9c86091"
server_key_iv: "c13a9e7a5672adadd5ed623066ca4849"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.229880 6479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:08.243610 6479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:08.245512 6479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:08.245642 6479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:08.249993 6479 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:38591
I20260927 07:09:08.250046 6556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:38591 every 8 connection(s)
I20260927 07:09:08.250363 6479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:08.251042 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:08.253366 6557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: Bootstrap starting.
I20260927 07:09:08.253939 6557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:08.254204 6557 log.cc:824] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:08.254963 6557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: No bootstrap required, opened a new log
I20260927 07:09:08.257033 6557 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac56a3ba9cf24989b56c9da254b0c6c4" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38591 } }
I20260927 07:09:08.257189 6557 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:08.257211 6557 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac56a3ba9cf24989b56c9da254b0c6c4, State: Initialized, Role: FOLLOWER
I20260927 07:09:08.257351 6557 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac56a3ba9cf24989b56c9da254b0c6c4" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38591 } }
I20260927 07:09:08.257418 6557 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:08.257450 6557 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:08.257493 6557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:08.257872 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 6479
I20260927 07:09:08.257939 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:08.258085 4793 external_mini_cluster.cc:1428] Setting key c7dd4c0e1962ccd5a52687fbf3e24abb
I20260927 07:09:08.258152 6557 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac56a3ba9cf24989b56c9da254b0c6c4" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38591 } }
I20260927 07:09:08.258261 6557 leader_election.cc:302] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac56a3ba9cf24989b56c9da254b0c6c4; no voters:
I20260927 07:09:08.258430 6557 leader_election.cc:288] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:08.258469 6560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:08.259023 6560 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [term 1 LEADER]: Becoming Leader. State: Replica: ac56a3ba9cf24989b56c9da254b0c6c4, State: Running, Role: LEADER
I20260927 07:09:08.259202 6560 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac56a3ba9cf24989b56c9da254b0c6c4" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38591 } }
I20260927 07:09:08.259567 6560 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac56a3ba9cf24989b56c9da254b0c6c4. Latest consensus state: current_term: 1 leader_uuid: "ac56a3ba9cf24989b56c9da254b0c6c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac56a3ba9cf24989b56c9da254b0c6c4" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38591 } } }
I20260927 07:09:08.259800 6560 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:08.259735 6563 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac56a3ba9cf24989b56c9da254b0c6c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac56a3ba9cf24989b56c9da254b0c6c4" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38591 } } }
I20260927 07:09:08.260174 6563 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:08.259020 6557 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:08.263037 6566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:08.265296 6566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:08.269675 6566 catalog_manager.cc:1347] Generated new cluster ID: 4be59882706b41cd89f3a78c7fa8717f
I20260927 07:09:08.269774 6566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:08.272178 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.258513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40440 (local address 127.4.174.126:38591)
0927 07:09:08.258680 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.258690 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.258736 (+ 46us) server_negotiation.cc:407] Connection header received
0927 07:09:08.258858 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.258866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.258927 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.259076 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.259942 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.260944 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.261664 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.262083 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.262177 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.268557 (+ 6380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.268561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.268562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.268575 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.270579 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.270940 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.270943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.270944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.271080 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.271261 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.271262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.271262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.271575 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.271741 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.271815 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.271955 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260927 07:09:08.274277 6566 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:08.275002 6566 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:08.278431 6566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: Generated new TSK 0
I20260927 07:09:08.278625 6566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:08.309289 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38591
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:08.396275 6583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:08.396482 6583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:08.396505 6583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:08.397658 6583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:08.397698 6583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:08.397717 6583 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 07:09:08.397727 6583 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:08.397743 6583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:08.399232 6583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:38591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:08.399459 6583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:08.399684 6583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:08.404592 6588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:08.404614 6589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:08.404805 6583 server_base.cc:1034] running on GCE node
W20260927 07:09:08.404609 6591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:08.405076 6583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:08.405345 6583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:08.406515 6583 hybrid_clock.cc:648] HybridClock initialized: now 1790492948406488 us; error 41 us; skew 500 ppm
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:08.409116 6583 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:08.409641 6583 webserver.cc:460] Webserver started at http://127.4.174.65:33601/ using document root <none> and password file <none>
I20260927 07:09:08.409870 6583 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:08.409919 6583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:08.410048 6583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:08.411180 6583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "a128f6769c414a8b8d87959fe7772459"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "625a41596a14fd2ad69aa534f9425966"
server_key_iv: "221c5e0cb4639e36a51b01e0d37109f5"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.411517 6583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "a128f6769c414a8b8d87959fe7772459"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "625a41596a14fd2ad69aa534f9425966"
server_key_iv: "221c5e0cb4639e36a51b01e0d37109f5"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.412791 6583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:08.413491 6598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.413652 6583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:08.413741 6583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "a128f6769c414a8b8d87959fe7772459"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "625a41596a14fd2ad69aa534f9425966"
server_key_iv: "221c5e0cb4639e36a51b01e0d37109f5"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.413806 6583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:08.436103 6583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:08.438393 6583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:08.438583 6583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:08.439152 6583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:08.439198 6583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.439232 6583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:08.439263 6583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:08.446615 6583 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:42741
I20260927 07:09:08.446974 6583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:08.447351 6713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:42741 every 8 connection(s)
I20260927 07:09:08.449218 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.439545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:34101 (local address 127.4.174.126:38591)
0927 07:09:08.439610 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.439613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.439861 (+ 248us) server_negotiation.cc:407] Connection header received
0927 07:09:08.440568 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.440569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.440584 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.440649 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.441537 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.442043 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.443233 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.443433 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.443489 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.446604 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.446605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.446605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.446616 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.447979 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.448337 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.448339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.448339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.448400 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.448606 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.448607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.448607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.448732 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.448787 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.449038 (+ 251us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.449128 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:09:08.449385 6607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.439533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38591 (local address 127.4.174.65:34101)
0927 07:09:08.439707 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.439760 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.440455 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.440743 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.440748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.441136 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:08.441439 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.441447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.442133 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.442134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.443135 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.443138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.443502 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.443503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.443632 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.444114 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:08.444123 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:08.446507 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:08.448052 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.448058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.448061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.448274 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.448451 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.448452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.448452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.448554 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.448786 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:08.448793 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:08.448924 (+ 131us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.449011 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.449162 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260927 07:09:08.449904 6714 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38591
I20260927 07:09:08.449991 6714 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:08.450170 6714 heartbeater.cc:511] Master 127.4.174.126:38591 requested a full tablet report, sending...
I20260927 07:09:08.450765 6508 ts_manager.cc:194] Registered new tserver with Master: a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741)
I20260927 07:09:08.451577 6508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:34101
I20260927 07:09:08.454689 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 6583
I20260927 07:09:08.454764 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:08.454934 4793 external_mini_cluster.cc:1428] Setting key 48706b73403ed700fcb08f1ed368734c
I20260927 07:09:08.455957 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.451038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:57609 (local address 127.4.174.126:38591)
0927 07:09:08.451083 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.451085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.451093 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:08.451114 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.451114 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.451125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.451195 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.451463 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.452097 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.452787 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.452969 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.453039 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.453806 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.453807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.453807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.453813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.454919 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.455253 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.455254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.455254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.455309 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.455467 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.455468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.455468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.455580 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.455638 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.455796 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.455868 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:09:08.455971 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.450854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38591 (local address 127.4.174.65:57609)
0927 07:09:08.450989 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.451000 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.451049 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.451206 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.451206 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.451346 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:08.451405 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.451406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.452176 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.452177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.452705 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.452707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.453032 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.453032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.453081 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.453451 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:08.453455 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:08.453735 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:08.454986 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.454987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.454988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.455186 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.455362 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.455363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.455363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.455422 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.455640 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:08.455641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:08.455689 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.455766 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.455867 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:08.477736 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38591
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:08.556226 6723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:08.556427 6723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:08.556449 6723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:08.557564 6723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:08.557602 6723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:08.557621 6723 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 07:09:08.557631 6723 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:08.557660 6723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:08.559118 6723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:38591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:08.559340 6723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:08.559559 6723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:08.564320 6728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:08.564371 6723 server_base.cc:1034] running on GCE node
W20260927 07:09:08.564471 6731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:08.564555 6729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:08.564764 6723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:08.564980 6723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:08.566113 6723 hybrid_clock.cc:648] HybridClock initialized: now 1790492948566100 us; error 28 us; skew 500 ppm
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:08.568447 6723 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:08.568856 6723 webserver.cc:460] Webserver started at http://127.4.174.66:39763/ using document root <none> and password file <none>
I20260927 07:09:08.569051 6723 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:08.569100 6723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:08.569226 6723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:08.570240 6723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "3e8b9119e8524d17a58137b6db876084"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "8c86b292899e61d99f0272a4c4b48dfd"
server_key_iv: "c70a086740ddbbeb9689746a0d33b57d"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.570592 6723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "3e8b9119e8524d17a58137b6db876084"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "8c86b292899e61d99f0272a4c4b48dfd"
server_key_iv: "c70a086740ddbbeb9689746a0d33b57d"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.571918 6723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:08.572741 6738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.572899 6723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:08.572983 6723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "3e8b9119e8524d17a58137b6db876084"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "8c86b292899e61d99f0272a4c4b48dfd"
server_key_iv: "c70a086740ddbbeb9689746a0d33b57d"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.573042 6723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:08.589869 6723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:08.592088 6723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:08.592221 6723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:08.592703 6723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:08.592753 6723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.592787 6723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:08.592808 6723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.598953 6723 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:36125
I20260927 07:09:08.599005 6853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:36125 every 8 connection(s)
I20260927 07:09:08.599310 6723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:08.602075 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 6723
I20260927 07:09:08.602172 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:08.602248 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.593208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:45585 (local address 127.4.174.126:38591)
0927 07:09:08.593258 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.593260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.593496 (+ 236us) server_negotiation.cc:407] Connection header received
0927 07:09:08.593994 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.593996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.594009 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.594077 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.594895 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.595640 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.597094 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.597389 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.597468 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.599940 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.599941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.599941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.599948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.600967 (+ 1019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.601324 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.601325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.601326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.601386 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.601595 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.601596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.601596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.601707 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.601763 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.602075 (+ 312us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.602157 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 07:09:08.602370 4793 external_mini_cluster.cc:1428] Setting key a6ac98b8a3b44bf3b528588eee9ea7d7
I20260927 07:09:08.602388 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.593205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38591 (local address 127.4.174.66:45585)
0927 07:09:08.593380 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.593417 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.593891 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.594150 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.594154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.594485 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:08.594792 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.594799 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.595733 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.595734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.597002 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.597004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.597496 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.597498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.597623 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.598186 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:08.598197 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:08.599844 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:08.601042 (+ 1198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.601048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.601050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.601255 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.601440 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.601441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.601441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.601539 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.601762 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:08.601769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:08.601905 (+ 136us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.602014 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.602176 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260927 07:09:08.603863 6854 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38591
I20260927 07:09:08.603951 6854 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:08.604082 6854 heartbeater.cc:511] Master 127.4.174.126:38591 requested a full tablet report, sending...
I20260927 07:09:08.604374 6511 ts_manager.cc:194] Registered new tserver with Master: 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125)
I20260927 07:09:08.604811 6511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:45585
I20260927 07:09:08.609256 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.603862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:47727 (local address 127.4.174.126:38591)
0927 07:09:08.604217 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.604219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.604227 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:08.604247 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.604248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.604257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.604321 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.604690 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.605283 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.606060 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.606260 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.606311 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.607129 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.607131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.607131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.607138 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.608275 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.608553 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.608554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.608554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.608603 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.608767 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.608768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.608768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.608903 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.608957 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.609092 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.609157 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332}
I20260927 07:09:08.609256 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.603819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38591 (local address 127.4.174.66:47727)
0927 07:09:08.603946 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.603956 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.604005 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.604374 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.604376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.604551 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:08.604619 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.604621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.605367 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.605368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.605990 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.605992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.606316 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.606317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.606386 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.606777 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:08.606780 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:08.607060 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:08.608343 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.608344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.608345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.608495 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.608655 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.608655 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.608656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.608712 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.608958 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:08.608959 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:08.609005 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.609063 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.609159 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":7,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:08.627879 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38591
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:08.706176 6863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:08.706379 6863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:08.706420 6863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:08.707633 6863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:08.707674 6863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:08.707702 6863 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 07:09:08.707716 6863 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:08.707741 6863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:08.709172 6863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:38591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:08.709399 6863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:08.709617 6863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:08.714231 6871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:08.714340 6868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:08.714390 6869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:08.714414 6863 server_base.cc:1034] running on GCE node
I20260927 07:09:08.714727 6863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:08.714953 6863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:08.716095 6863 hybrid_clock.cc:648] HybridClock initialized: now 1790492948716087 us; error 32 us; skew 500 ppm
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:08.718469 6863 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:08.718880 6863 webserver.cc:460] Webserver started at http://127.4.174.67:36221/ using document root <none> and password file <none>
I20260927 07:09:08.719084 6863 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:08.719131 6863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:08.719254 6863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:08.720232 6863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "21b18f719a8ed206d5f5087ab2093b39"
server_key_iv: "6e2a2acef7621e0b60e948a06a0a4ec4"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.720562 6863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "21b18f719a8ed206d5f5087ab2093b39"
server_key_iv: "6e2a2acef7621e0b60e948a06a0a4ec4"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.721803 6863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:08.722492 6878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.722646 6863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:08.722724 6863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82"
format_stamp: "Formatted at 2026-09-27 07:09:08 on dist-test-slave-zw7p"
server_key: "21b18f719a8ed206d5f5087ab2093b39"
server_key_iv: "6e2a2acef7621e0b60e948a06a0a4ec4"
server_key_version: "encryptionkey@0"
I20260927 07:09:08.722785 6863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:08.730355 6863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:08.732074 6863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:08.732223 6863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:08.732677 6863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:08.732717 6863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.732743 6863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:08.732759 6863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:08.739234 6863 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:39611
I20260927 07:09:08.739305 6993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:39611 every 8 connection(s)
I20260927 07:09:08.739625 6863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:08.741740 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.733113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:60155 (local address 127.4.174.126:38591)
0927 07:09:08.733183 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.733185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.733373 (+ 188us) server_negotiation.cc:407] Connection header received
0927 07:09:08.733871 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.733872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.733886 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.733946 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.734824 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.735331 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.736334 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.736539 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.736596 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.739138 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.739139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.739139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.739147 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.740538 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.740939 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.740941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.740941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.741008 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.741229 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.741231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.741231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.741352 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.741408 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.741613 (+ 205us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.741676 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 07:09:08.741889 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 6863
I20260927 07:09:08.741940 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.733103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38591 (local address 127.4.174.67:60155)
0927 07:09:08.733262 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.733301 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.733774 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.734045 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.734050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.734378 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:08.734709 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.734727 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.735431 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.735432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.736249 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.736251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.736631 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.736632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.736769 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.737249 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:08.737260 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:08.739008 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:08.740633 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.740636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.740638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.740854 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.741074 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.741075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.741075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.741162 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.741399 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:08.741403 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:08.741512 (+ 109us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.741588 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.741707 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260927 07:09:08.741986 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:08.742149 4793 external_mini_cluster.cc:1428] Setting key 0b9ba55bb0a4f82cffdf225098231113
I20260927 07:09:08.742676 6994 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38591
I20260927 07:09:08.742762 6994 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:08.742902 6994 heartbeater.cc:511] Master 127.4.174.126:38591 requested a full tablet report, sending...
I20260927 07:09:08.743216 6511 ts_manager.cc:194] Registered new tserver with Master: f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611)
I20260927 07:09:08.743708 6511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:60155
I20260927 07:09:08.749099 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.743637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38591 (local address 127.4.174.67:58309)
0927 07:09:08.743775 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.743785 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.743839 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.743970 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.743970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.744105 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:08.744159 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.744160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.744803 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.744803 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.745438 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.745440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.745808 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.745809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.745887 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.746595 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:08.746599 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:08.746959 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:08.748437 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.748437 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.748438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.748586 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.748721 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.748721 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.748722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.748762 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.748902 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:08.748902 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:08.748934 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.748974 (+ 40us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.749038 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260927 07:09:08.749100 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.743688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:58309 (local address 127.4.174.126:38591)
0927 07:09:08.743726 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.743728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.743827 (+ 99us) server_negotiation.cc:407] Connection header received
0927 07:09:08.743881 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.743882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.743891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.743955 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.744221 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.744735 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.745511 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.745747 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.745801 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.747043 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.747044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.747044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.747059 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.748374 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.748641 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.748641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.748641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.748683 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.748792 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.748793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.748793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.748871 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.748910 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.748990 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.749038 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:09:08.753748 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:08.774290 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.768208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.4.174.126:38591)
0927 07:09:08.768257 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.768259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.768302 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:09:08.768378 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.768378 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.768388 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.768433 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.768943 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.769444 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.770165 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.770347 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.770399 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.772465 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.772466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.772466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.772473 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.773492 (+ 1019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.773772 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.773773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.773773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.773814 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.773938 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.773939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.773939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.774027 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.774099 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.774168 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.774222 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:09:08.775897 6511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:08.777437 6511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:08.783502 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.779791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.4.174.66:36125)
0927 07:09:08.779942 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.779946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.779955 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:08.779994 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.779996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.780011 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.780084 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:08.780291 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.781044 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.782453 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.783180 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.783243 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.783292 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.783365 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.783403 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260927 07:09:08.783505 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.779442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42741 (local address 127.0.0.1:51176)
0927 07:09:08.779735 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.779745 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.779795 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.780014 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.780017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.780020 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:08.780083 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.780085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.781389 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.781390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.782444 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.782446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.783136 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.783137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.783229 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.783245 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.783310 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.783367 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":70,"threads_started":1}
I20260927 07:09:08.783598 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.779583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.4.174.65:42741)
0927 07:09:08.779766 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.779770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.779785 (+ 15us) server_negotiation.cc:407] Connection header received
0927 07:09:08.779899 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.779903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.779921 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.780001 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:08.780160 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.781309 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.782502 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.783051 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.783124 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.783186 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.783455 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.783494 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260927 07:09:08.783732 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.779424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39611 (local address 127.0.0.1:50612)
0927 07:09:08.779561 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.779601 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.779662 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.780105 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.780106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.780108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:08.780209 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.780211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.781423 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.781425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.782233 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.782236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.783253 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.783254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.783318 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.783322 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.783367 (+ 45us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.783405 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":44,"threads_started":1}
I20260927 07:09:08.783499 7010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.779602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36125 (local address 127.0.0.1:59150)
0927 07:09:08.779820 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.779829 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.779867 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.780151 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.780152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.780155 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:08.780205 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.780207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.781122 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.781122 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.782370 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.782373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.783241 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.783242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.783291 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.783295 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.783348 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.783376 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":22,"thread_start_us":50,"threads_started":1}
I20260927 07:09:08.784246 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.779572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50612 (local address 127.4.174.67:39611)
0927 07:09:08.779972 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.779975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.779983 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:08.780014 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.780016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.780028 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.780108 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:08.780300 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.781368 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.782317 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.783084 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.783150 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.783207 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.783546 (+ 339us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.784142 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":82,"threads_started":1}
I20260927 07:09:08.784269 6648 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 (DEFAULT_TABLE table=test-table [id=6974fda9d6fd4515a4c2cc30396a74ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:08.784634 6648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:08.784515 6788 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 (DEFAULT_TABLE table=test-table [id=6974fda9d6fd4515a4c2cc30396a74ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:08.784826 6788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:08.784803 6928 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 (DEFAULT_TABLE table=test-table [id=6974fda9d6fd4515a4c2cc30396a74ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:08.785085 6928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:08.787568 7014 tablet_bootstrap.cc:492] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Bootstrap starting.
I20260927 07:09:08.788105 7014 tablet_bootstrap.cc:654] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:08.788326 7015 tablet_bootstrap.cc:492] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Bootstrap starting.
I20260927 07:09:08.788439 7014 log.cc:824] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:08.789171 7014 tablet_bootstrap.cc:492] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: No bootstrap required, opened a new log
I20260927 07:09:08.789186 7015 tablet_bootstrap.cc:654] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:08.789253 7014 ts_tablet_manager.cc:1397] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:08.789508 7015 log.cc:824] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:08.789839 7016 tablet_bootstrap.cc:492] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Bootstrap starting.
I20260927 07:09:08.790176 7015 tablet_bootstrap.cc:492] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: No bootstrap required, opened a new log
I20260927 07:09:08.790235 7015 ts_tablet_manager.cc:1397] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:08.790405 7016 tablet_bootstrap.cc:654] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:08.790757 7016 log.cc:824] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:08.791334 7014 raft_consensus.cc:348] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.791460 7014 raft_consensus.cc:374] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:08.791481 7014 raft_consensus.cc:729] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8b9119e8524d17a58137b6db876084, State: Initialized, Role: FOLLOWER
I20260927 07:09:08.791574 7014 consensus_queue.cc:260] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.791813 7014 ts_tablet_manager.cc:1428] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 07:09:08.791826 7015 raft_consensus.cc:348] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.791931 6854 heartbeater.cc:503] Master 127.4.174.126:38591 was elected leader, sending a full tablet report...
I20260927 07:09:08.791975 7015 raft_consensus.cc:374] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:08.792009 7015 raft_consensus.cc:729] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b2f0c38dc14d4f9c491ee3d6cb1d82, State: Initialized, Role: FOLLOWER
I20260927 07:09:08.792146 7015 consensus_queue.cc:260] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.792408 7015 ts_tablet_manager.cc:1428] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:09:08.792431 7016 tablet_bootstrap.cc:492] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: No bootstrap required, opened a new log
I20260927 07:09:08.792429 6994 heartbeater.cc:503] Master 127.4.174.126:38591 was elected leader, sending a full tablet report...
I20260927 07:09:08.792487 7016 ts_tablet_manager.cc:1397] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260927 07:09:08.794000 7016 raft_consensus.cc:348] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.794117 7016 raft_consensus.cc:374] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:08.794147 7016 raft_consensus.cc:729] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a128f6769c414a8b8d87959fe7772459, State: Initialized, Role: FOLLOWER
I20260927 07:09:08.794255 7016 consensus_queue.cc:260] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.794462 7016 ts_tablet_manager.cc:1428] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:09:08.794525 6714 heartbeater.cc:503] Master 127.4.174.126:38591 was elected leader, sending a full tablet report...
W20260927 07:09:08.849978 6855 tablet.cc:2369] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:08.941797 7021 raft_consensus.cc:482] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:08.941931 7021 raft_consensus.cc:504] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.942301 7021 leader_election.cc:288] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125), a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741)
I20260927 07:09:08.945123 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.942588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:50369 (local address 127.4.174.66:36125)
0927 07:09:08.942671 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.942673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.942682 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:08.942715 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.942716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.942725 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.942810 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:08.942959 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.943487 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.944468 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.944813 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.944865 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.944873 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.945015 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.945054 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:09:08.945145 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.942607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:56623 (local address 127.4.174.65:42741)
0927 07:09:08.942653 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.942655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.942818 (+ 163us) server_negotiation.cc:407] Connection header received
0927 07:09:08.942867 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.942867 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.942878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.942926 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:08.943074 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.943515 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.944468 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.944838 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.944902 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.944910 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.945041 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.945078 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:08.945147 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.942499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36125 (local address 127.4.174.67:50369)
0927 07:09:08.942573 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.942584 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.942629 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.942820 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.942821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.942823 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:08.942896 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.942898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.943561 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.943562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.944381 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.944383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.944878 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.944879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.944953 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.944957 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.944994 (+ 37us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.945028 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":49}
I20260927 07:09:08.945165 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.942499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42741 (local address 127.4.174.67:56623)
0927 07:09:08.942744 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.942755 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.942813 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.942940 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.942941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.942942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:08.943013 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.943015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.943586 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.943587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.944386 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.944388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.944909 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.944910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.944977 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.944981 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.945031 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.945062 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":86,"thread_start_us":88,"threads_started":1}
I20260927 07:09:08.945577 6668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebdd7ddcbca4aaf9786e2d27cf41540" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a128f6769c414a8b8d87959fe7772459" is_pre_election: true
I20260927 07:09:08.945586 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebdd7ddcbca4aaf9786e2d27cf41540" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084" is_pre_election: true
I20260927 07:09:08.945706 6668 raft_consensus.cc:2454] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 0.
I20260927 07:09:08.945725 6808 raft_consensus.cc:2454] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 0.
I20260927 07:09:08.945909 6882 leader_election.cc:302] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a128f6769c414a8b8d87959fe7772459, f2b2f0c38dc14d4f9c491ee3d6cb1d82; no voters:
I20260927 07:09:08.946075 7021 raft_consensus.cc:2781] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:08.946116 7021 raft_consensus.cc:482] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:08.946130 7021 raft_consensus.cc:3037] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:08.947060 7021 raft_consensus.cc:504] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.947175 7021 leader_election.cc:288] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125), a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741)
I20260927 07:09:08.947296 6668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebdd7ddcbca4aaf9786e2d27cf41540" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a128f6769c414a8b8d87959fe7772459"
I20260927 07:09:08.947337 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebdd7ddcbca4aaf9786e2d27cf41540" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084"
I20260927 07:09:08.947371 6668 raft_consensus.cc:3037] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:08.947396 6808 raft_consensus.cc:3037] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Advancing to term 1
W20260927 07:09:08.947870 6715 tablet.cc:2369] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:08.948101 6808 raft_consensus.cc:2454] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 1.
I20260927 07:09:08.948124 6668 raft_consensus.cc:2454] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 1.
I20260927 07:09:08.948257 6881 leader_election.cc:302] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8b9119e8524d17a58137b6db876084, f2b2f0c38dc14d4f9c491ee3d6cb1d82; no voters:
I20260927 07:09:08.948342 7021 raft_consensus.cc:2781] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:08.948472 7021 raft_consensus.cc:686] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 LEADER]: Becoming Leader. State: Replica: f2b2f0c38dc14d4f9c491ee3d6cb1d82, State: Running, Role: LEADER
I20260927 07:09:08.948552 7021 consensus_queue.cc:237] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } }
I20260927 07:09:08.949357 6510 catalog_manager.cc:5495] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67). New cstate: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } health_report { overall_health: UNKNOWN } } }
Sep 27 07:09:08 dist-test-slave-zw7p krb5kdc[6466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492948, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:08.981217 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.975217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40446 (local address 127.4.174.126:38591)
0927 07:09:08.975275 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:08.975277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:08.975382 (+ 105us) server_negotiation.cc:407] Connection header received
0927 07:09:08.975447 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:08.975448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:08.975457 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:08.975496 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:08.975948 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.976440 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.977082 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.977267 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.977319 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.979177 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:08.979179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:08.979179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:08.979185 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:08.980229 (+ 1044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.980545 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.980546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.980546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.980591 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:08.980744 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:08.980745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:08.980746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:08.980840 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:08.980892 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:08.981026 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 07:09:08.981110 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":29}
I20260927 07:09:08.981279 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:08.975156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38591 (local address 127.0.0.1:40446)
0927 07:09:08.975312 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:08.975324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:08.975392 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:08.975509 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:08.975509 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:08.975817 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:08.975859 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.975861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.976502 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.976503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.977016 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:08.977017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:08.977322 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:08.977323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:08.977372 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:08.977839 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:08.977842 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:08.979100 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:08.980299 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.980308 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.980310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.980486 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.980641 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:08.980642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:08.980642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:08.980692 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:08.980891 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:08.980899 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:08.980943 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 07:09:08.980999 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 07:09:08.981175 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":78,"thread_start_us":87,"threads_started":1}
I20260927 07:09:08.983026 6510 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:40446:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 07:09:08.983222 6510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:08.984516 6788 tablet_service.cc:1467] Processing CreateTablet for tablet e13bd4b3807442ffaf0e5166d018044f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b23ed522d0cb488c8c286cce6989aa39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:08.984488 6648 tablet_service.cc:1467] Processing CreateTablet for tablet e13bd4b3807442ffaf0e5166d018044f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b23ed522d0cb488c8c286cce6989aa39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:08.984529 6928 tablet_service.cc:1467] Processing CreateTablet for tablet e13bd4b3807442ffaf0e5166d018044f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b23ed522d0cb488c8c286cce6989aa39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:08.984650 6788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e13bd4b3807442ffaf0e5166d018044f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:08.984658 6928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e13bd4b3807442ffaf0e5166d018044f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:08.984658 6648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e13bd4b3807442ffaf0e5166d018044f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:08.985657 7014 tablet_bootstrap.cc:492] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: Bootstrap starting.
I20260927 07:09:08.986040 7016 tablet_bootstrap.cc:492] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: Bootstrap starting.
I20260927 07:09:08.986289 7014 tablet_bootstrap.cc:654] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:08.986357 7016 tablet_bootstrap.cc:654] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:08.986675 7015 tablet_bootstrap.cc:492] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Bootstrap starting.
I20260927 07:09:08.986961 7014 tablet_bootstrap.cc:492] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: No bootstrap required, opened a new log
I20260927 07:09:08.987011 7014 ts_tablet_manager.cc:1397] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:08.987072 7015 tablet_bootstrap.cc:654] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:08.987154 7014 raft_consensus.cc:348] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:08.987251 7014 raft_consensus.cc:374] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:08.987277 7014 raft_consensus.cc:729] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8b9119e8524d17a58137b6db876084, State: Initialized, Role: FOLLOWER
I20260927 07:09:08.987327 7014 consensus_queue.cc:260] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:08.987419 7014 ts_tablet_manager.cc:1428] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:08.987533 7016 tablet_bootstrap.cc:492] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: No bootstrap required, opened a new log
I20260927 07:09:08.987581 7016 ts_tablet_manager.cc:1397] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:08.987552 7020 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:08.987619 7020 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:08.987759 7016 raft_consensus.cc:348] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:08.987830 7016 raft_consensus.cc:374] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:08.987848 7016 raft_consensus.cc:729] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a128f6769c414a8b8d87959fe7772459, State: Initialized, Role: FOLLOWER
I20260927 07:09:08.987893 7016 consensus_queue.cc:260] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:08.987977 7016 ts_tablet_manager.cc:1428] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.988022 7022 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:08.988121 7022 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:08.988309 7015 tablet_bootstrap.cc:492] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: No bootstrap required, opened a new log
I20260927 07:09:08.988358 7015 ts_tablet_manager.cc:1397] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 07:09:08.988495 7015 raft_consensus.cc:348] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:08.988567 7015 raft_consensus.cc:374] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:08.988586 7015 raft_consensus.cc:729] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b2f0c38dc14d4f9c491ee3d6cb1d82, State: Initialized, Role: FOLLOWER
I20260927 07:09:08.988632 7015 consensus_queue.cc:260] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:08.988729 7015 ts_tablet_manager.cc:1428] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:08.988765 7021 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:08.988849 7021 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: This TxnStatusTablet replica's current role is: FOLLOWER
W20260927 07:09:08.990139 6995 tablet.cc:2369] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:09.223409 7021 raft_consensus.cc:482] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:09.223495 7021 raft_consensus.cc:504] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.223642 7021 leader_election.cc:288] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741), 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125)
I20260927 07:09:09.223907 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e13bd4b3807442ffaf0e5166d018044f" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084" is_pre_election: true
I20260927 07:09:09.223907 6668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e13bd4b3807442ffaf0e5166d018044f" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a128f6769c414a8b8d87959fe7772459" is_pre_election: true
I20260927 07:09:09.224031 6808 raft_consensus.cc:2454] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 0.
I20260927 07:09:09.224031 6668 raft_consensus.cc:2454] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 0.
I20260927 07:09:09.224206 6881 leader_election.cc:302] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3e8b9119e8524d17a58137b6db876084, a128f6769c414a8b8d87959fe7772459, f2b2f0c38dc14d4f9c491ee3d6cb1d82; no voters:
I20260927 07:09:09.224320 7021 raft_consensus.cc:2781] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:09.224368 7021 raft_consensus.cc:482] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:09.224393 7021 raft_consensus.cc:3037] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:09.225057 7021 raft_consensus.cc:504] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.225183 7021 leader_election.cc:288] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 election: Requested vote from peers a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741), 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125)
I20260927 07:09:09.225351 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e13bd4b3807442ffaf0e5166d018044f" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084"
I20260927 07:09:09.225360 6668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e13bd4b3807442ffaf0e5166d018044f" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a128f6769c414a8b8d87959fe7772459"
I20260927 07:09:09.225430 6808 raft_consensus.cc:3037] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:09.225430 6668 raft_consensus.cc:3037] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:09.226019 6668 raft_consensus.cc:2454] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 1.
I20260927 07:09:09.226114 6808 raft_consensus.cc:2454] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 1.
I20260927 07:09:09.226214 6882 leader_election.cc:302] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a128f6769c414a8b8d87959fe7772459, f2b2f0c38dc14d4f9c491ee3d6cb1d82; no voters:
I20260927 07:09:09.226353 7021 raft_consensus.cc:2781] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:09.226410 7021 raft_consensus.cc:686] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 LEADER]: Becoming Leader. State: Replica: f2b2f0c38dc14d4f9c491ee3d6cb1d82, State: Running, Role: LEADER
I20260927 07:09:09.226464 7021 consensus_queue.cc:237] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.226802 7024 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: TxnStatusTablet state changed. Reason: New leader f2b2f0c38dc14d4f9c491ee3d6cb1d82. Latest consensus state: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:09.226891 7024 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:09.227113 7036 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 07:09:09.227123 6510 catalog_manager.cc:5495] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67). New cstate: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } health_report { overall_health: HEALTHY } } }
I20260927 07:09:09.227228 7036 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 07:09:09.272990 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.270076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50622 (local address 127.4.174.67:39611)
0927 07:09:09.270139 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:09.270141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:09.270190 (+ 49us) server_negotiation.cc:407] Connection header received
0927 07:09:09.270276 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:09.270277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:09.270297 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:09.270366 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:09.270467 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.271244 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.272040 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.272347 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.272414 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.272644 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.272771 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.272848 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:09.272886 (+ 38us) server_negotiation.cc:299] Negotiation successful
0927 07:09:09.272906 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 07:09:09.272990 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.270001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39611 (local address 127.0.0.1:50622)
0927 07:09:09.270143 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:09.270153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:09.270201 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:09.270368 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:09.270369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:09.270370 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:09.270416 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.270418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.271345 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.271347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.271954 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.271956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.272427 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.272428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.272533 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.272552 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.272835 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.272844 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 07:09:09.272882 (+ 38us) client_negotiation.cc:240] Negotiation successful
0927 07:09:09.272927 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260927 07:09:09.274984 6808 raft_consensus.cc:1261] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Refusing update from remote peer f2b2f0c38dc14d4f9c491ee3d6cb1d82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:09.275018 6668 raft_consensus.cc:1261] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Refusing update from remote peer f2b2f0c38dc14d4f9c491ee3d6cb1d82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:09.275321 7021 consensus_queue.cc:1035] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:09.275457 7024 consensus_queue.cc:1035] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:09.276620 7022 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: TxnStatusTablet state changed. Reason: New leader f2b2f0c38dc14d4f9c491ee3d6cb1d82. Latest consensus state: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:09.276705 7022 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:09.277468 7024 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:09.277559 7024 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:09.277853 7040 mvcc.cc:205] Tried to move back new op lower bound from 7333859120226971648 to 7333859120032247808. Current Snapshot: MvccSnapshot[applied={T|T < 7333859120226971648}]
I20260927 07:09:09.277851 7020 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: TxnStatusTablet state changed. Reason: New leader f2b2f0c38dc14d4f9c491ee3d6cb1d82. Latest consensus state: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:09.277938 7020 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:09.278297 7022 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:09.278363 7022 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P a128f6769c414a8b8d87959fe7772459: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:09.279348 7020 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:09.279438 7020 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P 3e8b9119e8524d17a58137b6db876084: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:09.279062 7039 mvcc.cc:205] Tried to move back new op lower bound from 7333859120226971648 to 7333859120032247808. Current Snapshot: MvccSnapshot[applied={T|T < 7333859120226971648 or (T in {7333859120226971648})}]
I20260927 07:09:09.280015 7021 tablet_replica.cc:439] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } } }
I20260927 07:09:09.280107 7021 tablet_replica.cc:442] T e13bd4b3807442ffaf0e5166d018044f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:09.280247 7043 mvcc.cc:205] Tried to move back new op lower bound from 7333859120226971648 to 7333859120032247808. Current Snapshot: MvccSnapshot[applied={T|T < 7333859120226971648}]
I20260927 07:09:09.283301 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.280813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50638 (local address 127.4.174.67:39611)
0927 07:09:09.280845 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:09.280848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:09.280855 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:09.280901 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:09.280901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:09.280910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:09.280968 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:09.281084 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.281808 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.282631 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.282894 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.282953 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.283098 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.283131 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.283175 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:09.283222 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 07:09:09.283241 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:09:09.289065 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.285679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:36493 (local address 127.4.174.67:39611)
0927 07:09:09.285713 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:09.285715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:09.286722 (+ 1007us) server_negotiation.cc:407] Connection header received
0927 07:09:09.286747 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:09.286748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:09.286759 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:09.286819 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:09.286899 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.287570 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.288253 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.288531 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.288602 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.288783 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.288813 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.288861 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:09.288945 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 07:09:09.288964 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":17}
I20260927 07:09:09.289079 7051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.285576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39611 (local address 127.4.174.67:36493)
0927 07:09:09.286655 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:09.286664 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:09.286716 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:09.286821 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:09.286821 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:09.286823 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:09.286861 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.286862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.287637 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.287637 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.288182 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.288183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.288634 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.288635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.288683 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.288697 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.288866 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.288867 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 07:09:09.288940 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 07:09:09.288988 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260927 07:09:09.291626 6668 raft_consensus.cc:1261] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Refusing update from remote peer f2b2f0c38dc14d4f9c491ee3d6cb1d82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:09.291652 6808 raft_consensus.cc:1261] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Refusing update from remote peer f2b2f0c38dc14d4f9c491ee3d6cb1d82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:09.291831 7052 consensus_queue.cc:1035] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:09.291946 7024 consensus_queue.cc:1035] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:09.306218 6511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:09.306352 6511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:09.307117 6511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: Sending DeleteTablet for 3 replicas of tablet 9ebdd7ddcbca4aaf9786e2d27cf41540
I20260927 07:09:09.307420 6788 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:09 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:59150
I20260927 07:09:09.307447 6928 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:09 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50612
I20260927 07:09:09.307579 6648 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:09 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:51176
I20260927 07:09:09.307842 7056 tablet_replica.cc:330] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: stopping tablet replica
I20260927 07:09:09.307865 6511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 07:09:09.307969 7058 tablet_replica.cc:330] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: stopping tablet replica
W20260927 07:09:09.307987 6511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:09.308001 7056 raft_consensus.cc:2229] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:09.308060 7056 raft_consensus.cc:2258] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:09.308075 7058 raft_consensus.cc:2229] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:09.308130 7058 raft_consensus.cc:2258] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:09.308538 7058 ts_tablet_manager.cc:1905] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:09.309111 6788 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2fc5dd735e4beb8b36daa2a2648541 (DEFAULT_TABLE table=test-table [id=0ec45938a2ab4e739c98dad7cafd17d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:09.309165 6648 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2fc5dd735e4beb8b36daa2a2648541 (DEFAULT_TABLE table=test-table [id=0ec45938a2ab4e739c98dad7cafd17d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:09.309232 6788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2fc5dd735e4beb8b36daa2a2648541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:09.309242 6648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2fc5dd735e4beb8b36daa2a2648541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:09.310447 7014 tablet_bootstrap.cc:492] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: Bootstrap starting.
I20260927 07:09:09.310479 7016 tablet_bootstrap.cc:492] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: Bootstrap starting.
I20260927 07:09:09.310798 6928 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2fc5dd735e4beb8b36daa2a2648541 (DEFAULT_TABLE table=test-table [id=0ec45938a2ab4e739c98dad7cafd17d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:09.310878 7014 tablet_bootstrap.cc:654] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:09.310878 7016 tablet_bootstrap.cc:654] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:09.310907 6928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2fc5dd735e4beb8b36daa2a2648541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:09.310381 7058 ts_tablet_manager.cc:1918] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:09.311383 7058 log.cc:1196] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/9ebdd7ddcbca4aaf9786e2d27cf41540
I20260927 07:09:09.311501 7014 tablet_bootstrap.cc:492] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: No bootstrap required, opened a new log
I20260927 07:09:09.311548 7014 ts_tablet_manager.cc:1397] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:09.311702 7058 ts_tablet_manager.cc:1939] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P a128f6769c414a8b8d87959fe7772459: Deleting consensus metadata
I20260927 07:09:09.311697 7014 raft_consensus.cc:348] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.311760 7014 raft_consensus.cc:374] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:09.311779 7014 raft_consensus.cc:729] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8b9119e8524d17a58137b6db876084, State: Initialized, Role: FOLLOWER
I20260927 07:09:09.311816 7014 consensus_queue.cc:260] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.311890 7014 ts_tablet_manager.cc:1428] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:09.312107 6498 catalog_manager.cc:4841] TS a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741): tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 (table test-table [id=6974fda9d6fd4515a4c2cc30396a74ea]) successfully deleted
I20260927 07:09:09.312206 7015 tablet_bootstrap.cc:492] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Bootstrap starting.
I20260927 07:09:09.312294 7057 tablet_replica.cc:330] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: stopping tablet replica
I20260927 07:09:09.312399 7057 raft_consensus.cc:2229] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:09.312493 7057 raft_consensus.cc:2258] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:09.312577 7015 tablet_bootstrap.cc:654] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:09.312610 7056 ts_tablet_manager.cc:1905] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:09.312904 7057 ts_tablet_manager.cc:1905] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:09.313355 7016 tablet_bootstrap.cc:492] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: No bootstrap required, opened a new log
I20260927 07:09:09.313401 7016 ts_tablet_manager.cc:1397] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 07:09:09.313542 7016 raft_consensus.cc:348] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.313621 7016 raft_consensus.cc:374] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:09.313640 7016 raft_consensus.cc:729] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a128f6769c414a8b8d87959fe7772459, State: Initialized, Role: FOLLOWER
I20260927 07:09:09.313680 7016 consensus_queue.cc:260] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.313762 7016 ts_tablet_manager.cc:1428] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:09.314073 7056 ts_tablet_manager.cc:1918] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:09.314174 7056 log.cc:1196] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/9ebdd7ddcbca4aaf9786e2d27cf41540
I20260927 07:09:09.314376 7057 ts_tablet_manager.cc:1918] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:09.314457 7056 ts_tablet_manager.cc:1939] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P 3e8b9119e8524d17a58137b6db876084: Deleting consensus metadata
I20260927 07:09:09.314479 7057 log.cc:1196] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/9ebdd7ddcbca4aaf9786e2d27cf41540
I20260927 07:09:09.314755 7057 ts_tablet_manager.cc:1939] T 9ebdd7ddcbca4aaf9786e2d27cf41540 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting consensus metadata
I20260927 07:09:09.314867 7015 tablet_bootstrap.cc:492] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: No bootstrap required, opened a new log
I20260927 07:09:09.314906 7015 ts_tablet_manager.cc:1397] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260927 07:09:09.314905 6497 catalog_manager.cc:4841] TS 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125): tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 (table test-table [id=6974fda9d6fd4515a4c2cc30396a74ea]) successfully deleted
I20260927 07:09:09.315047 7015 raft_consensus.cc:348] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.315110 7015 raft_consensus.cc:374] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:09.315127 7015 raft_consensus.cc:729] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b2f0c38dc14d4f9c491ee3d6cb1d82, State: Initialized, Role: FOLLOWER
I20260927 07:09:09.315132 6497 catalog_manager.cc:4841] TS f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611): tablet 9ebdd7ddcbca4aaf9786e2d27cf41540 (table test-table [id=6974fda9d6fd4515a4c2cc30396a74ea]) successfully deleted
I20260927 07:09:09.315164 7015 consensus_queue.cc:260] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.315245 7015 ts_tablet_manager.cc:1428] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:09.413367 7022 raft_consensus.cc:482] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:09.413467 7022 raft_consensus.cc:504] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.413789 7022 leader_election.cc:288] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125), f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611)
I20260927 07:09:09.417240 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.414049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:40783 (local address 127.4.174.66:36125)
0927 07:09:09.414114 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:09.414116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:09.414202 (+ 86us) server_negotiation.cc:407] Connection header received
0927 07:09:09.414313 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:09.414314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:09.414323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:09.414353 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:09.414665 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.415389 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.416593 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.416935 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.416993 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.417002 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:09.417139 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 07:09:09.417176 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:09.417250 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.413968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36125 (local address 127.4.174.65:40783)
0927 07:09:09.414138 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:09.414148 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:09.414235 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:09.414421 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:09.414422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:09.414424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:09.414489 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.414491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.415478 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.415479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.416496 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.416497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.417007 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.417008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.417080 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.417083 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:09.417125 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 07:09:09.417161 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260927 07:09:09.417562 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2fc5dd735e4beb8b36daa2a2648541" candidate_uuid: "a128f6769c414a8b8d87959fe7772459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084" is_pre_election: true
I20260927 07:09:09.417625 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.414268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:44675 (local address 127.4.174.67:39611)
0927 07:09:09.414311 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:09.414313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:09.414362 (+ 49us) server_negotiation.cc:407] Connection header received
0927 07:09:09.414418 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:09.414418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:09.414429 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:09.414510 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:09.414689 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.415153 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.416574 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.417133 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.417231 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.417242 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:09.417510 (+ 268us) server_negotiation.cc:299] Negotiation successful
0927 07:09:09.417554 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:09:09.417652 6808 raft_consensus.cc:2454] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a128f6769c414a8b8d87959fe7772459 in term 0.
I20260927 07:09:09.417824 6601 leader_election.cc:302] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8b9119e8524d17a58137b6db876084, a128f6769c414a8b8d87959fe7772459; no voters:
I20260927 07:09:09.417924 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.414190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39611 (local address 127.4.174.65:44675)
0927 07:09:09.414299 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:09.414314 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:09.414365 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:09.414516 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:09.414516 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:09.414518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:09.414598 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.414600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.415227 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.415228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.416484 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.416487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.417300 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.417301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.417381 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.417385 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:09.417748 (+ 363us) client_negotiation.cc:240] Negotiation successful
0927 07:09:09.417791 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260927 07:09:09.417950 7022 raft_consensus.cc:2781] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:09.417980 7022 raft_consensus.cc:482] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:09.418000 7022 raft_consensus.cc:3037] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:09.418711 7022 raft_consensus.cc:504] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.418833 7022 leader_election.cc:288] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125), f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611)
I20260927 07:09:09.418826 6948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2fc5dd735e4beb8b36daa2a2648541" candidate_uuid: "a128f6769c414a8b8d87959fe7772459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" is_pre_election: true
I20260927 07:09:09.418979 6948 raft_consensus.cc:2454] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a128f6769c414a8b8d87959fe7772459 in term 0.
I20260927 07:09:09.419023 6947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2fc5dd735e4beb8b36daa2a2648541" candidate_uuid: "a128f6769c414a8b8d87959fe7772459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82"
I20260927 07:09:09.419118 6947 raft_consensus.cc:3037] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:09.419690 6947 raft_consensus.cc:2454] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a128f6769c414a8b8d87959fe7772459 in term 1.
I20260927 07:09:09.419839 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2fc5dd735e4beb8b36daa2a2648541" candidate_uuid: "a128f6769c414a8b8d87959fe7772459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084"
I20260927 07:09:09.419909 6808 raft_consensus.cc:3037] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:09.420444 6808 raft_consensus.cc:2454] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a128f6769c414a8b8d87959fe7772459 in term 1.
I20260927 07:09:09.420598 6601 leader_election.cc:302] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a128f6769c414a8b8d87959fe7772459, f2b2f0c38dc14d4f9c491ee3d6cb1d82; no voters:
I20260927 07:09:09.420686 7022 raft_consensus.cc:2781] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:09.420835 7022 raft_consensus.cc:686] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 1 LEADER]: Becoming Leader. State: Replica: a128f6769c414a8b8d87959fe7772459, State: Running, Role: LEADER
I20260927 07:09:09.420910 7022 consensus_queue.cc:237] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:09.421681 6511 catalog_manager.cc:5495] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 reported cstate change: term changed from 0 to 1, leader changed from <none> to a128f6769c414a8b8d87959fe7772459 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "a128f6769c414a8b8d87959fe7772459" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:09.431265 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.428612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.4.174.65:42741)
0927 07:09:09.428651 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:09.428653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:09.428697 (+ 44us) server_negotiation.cc:407] Connection header received
0927 07:09:09.428773 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:09.428774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:09.428792 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:09.428851 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:09.428982 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.429445 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.430487 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.430740 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.430793 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.430965 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.431076 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.431126 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:09.431195 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 07:09:09.431210 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 07:09:09.436400 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.433850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:34887 (local address 127.4.174.67:39611)
0927 07:09:09.433891 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:09.433893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:09.434019 (+ 126us) server_negotiation.cc:407] Connection header received
0927 07:09:09.434071 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:09.434071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:09.434081 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:09.434123 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:09.434264 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.434750 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.435813 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.435995 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.436040 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.436223 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.436245 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.436276 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:09.436332 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 07:09:09.436346 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:09:09.436481 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.433793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39611 (local address 127.4.174.65:34887)
0927 07:09:09.433953 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:09.433963 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:09.434025 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:09.434127 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:09.434128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:09.434129 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:09.434195 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.434196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.434819 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.434820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.435735 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.435738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.436061 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.436062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.436130 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.436146 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.436282 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.436283 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 07:09:09.436330 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 07:09:09.436371 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260927 07:09:09.438480 6808 raft_consensus.cc:1261] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Refusing update from remote peer a128f6769c414a8b8d87959fe7772459: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:09.438582 6947 raft_consensus.cc:1261] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Refusing update from remote peer a128f6769c414a8b8d87959fe7772459: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:09.438737 7064 consensus_queue.cc:1035] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:09.438850 7022 consensus_queue.cc:1035] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:09.445313 7051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.443248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42741 (local address 127.4.174.67:43997)
0927 07:09:09.443290 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:09.443300 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:09.443331 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:09.443509 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:09.443510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:09.443511 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:09.443547 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.443549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.444112 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.444113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.444642 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:09.444643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.444953 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.444953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:09.445005 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.445011 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.445164 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.445165 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 07:09:09.445204 (+ 39us) client_negotiation.cc:240] Negotiation successful
0927 07:09:09.445248 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260927 07:09:09.445304 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:09.443345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:43997 (local address 127.4.174.65:42741)
0927 07:09:09.443394 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:09.443396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:09.443402 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:09:09.443419 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:09.443420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:09.443429 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:09.443478 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:09.443595 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.444045 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.444708 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:09.444901 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:09.444950 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:09.445066 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:09.445101 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:09.445160 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:09.445218 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 07:09:09.445237 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260927 07:09:09.445847 7040 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 07:09:10.971287 6511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:10.971386 6511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:10.972111 6511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: Sending DeleteTablet for 3 replicas of tablet 3e2fc5dd735e4beb8b36daa2a2648541
I20260927 07:09:10.972321 6648 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e2fc5dd735e4beb8b36daa2a2648541 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:10 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:51176
I20260927 07:09:10.972361 6927 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e2fc5dd735e4beb8b36daa2a2648541 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:10 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50612
I20260927 07:09:10.972362 6788 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e2fc5dd735e4beb8b36daa2a2648541 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:10 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:59150
I20260927 07:09:10.972644 7093 tablet_replica.cc:330] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: stopping tablet replica
I20260927 07:09:10.972702 7093 raft_consensus.cc:2229] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:10.972800 7093 raft_consensus.cc:2258] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:10.972913 7095 tablet_replica.cc:330] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: stopping tablet replica
I20260927 07:09:10.972968 7095 raft_consensus.cc:2229] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:10.973038 7093 ts_tablet_manager.cc:1905] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:10.973062 7094 tablet_replica.cc:330] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: stopping tablet replica
I20260927 07:09:10.973109 7094 raft_consensus.cc:2229] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:10.973165 7094 raft_consensus.cc:2258] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:10.973228 7095 raft_consensus.cc:2258] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:10.973421 7095 ts_tablet_manager.cc:1905] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:10.973529 7094 ts_tablet_manager.cc:1905] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:10.974661 7093 ts_tablet_manager.cc:1918] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:09:10.974754 7093 log.cc:1196] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/3e2fc5dd735e4beb8b36daa2a2648541
I20260927 07:09:10.975018 7093 ts_tablet_manager.cc:1939] T 3e2fc5dd735e4beb8b36daa2a2648541 P a128f6769c414a8b8d87959fe7772459: Deleting consensus metadata
I20260927 07:09:10.975003 7095 ts_tablet_manager.cc:1918] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:09:10.975075 7095 log.cc:1196] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/3e2fc5dd735e4beb8b36daa2a2648541
I20260927 07:09:10.975250 7095 ts_tablet_manager.cc:1939] T 3e2fc5dd735e4beb8b36daa2a2648541 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting consensus metadata
I20260927 07:09:10.975339 6498 catalog_manager.cc:4841] TS a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741): tablet 3e2fc5dd735e4beb8b36daa2a2648541 (table test-table [id=0ec45938a2ab4e739c98dad7cafd17d5]) successfully deleted
I20260927 07:09:10.975395 7094 ts_tablet_manager.cc:1918] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:09:10.975437 7094 log.cc:1196] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/3e2fc5dd735e4beb8b36daa2a2648541
I20260927 07:09:10.975579 6497 catalog_manager.cc:4841] TS f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611): tablet 3e2fc5dd735e4beb8b36daa2a2648541 (table test-table [id=0ec45938a2ab4e739c98dad7cafd17d5]) successfully deleted
I20260927 07:09:10.975636 7094 ts_tablet_manager.cc:1939] T 3e2fc5dd735e4beb8b36daa2a2648541 P 3e8b9119e8524d17a58137b6db876084: Deleting consensus metadata
I20260927 07:09:10.975869 6497 catalog_manager.cc:4841] TS 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125): tablet 3e2fc5dd735e4beb8b36daa2a2648541 (table test-table [id=0ec45938a2ab4e739c98dad7cafd17d5]) successfully deleted
Sep 27 07:09:10 dist-test-slave-zw7p krb5kdc[6466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:09:10 dist-test-slave-zw7p krb5kdc[6466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:10.995867 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:10.988938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40456 (local address 127.4.174.126:38591)
0927 07:09:10.989093 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:10.989095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:10.989164 (+ 69us) server_negotiation.cc:407] Connection header received
0927 07:09:10.989246 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:10.989246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:10.989258 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:10.989326 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:10.989922 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:10.990435 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:10.991235 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:10.991445 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:10.991511 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:10.993694 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:10.993696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:10.993696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:10.993703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:10.994956 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:10.995253 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:10.995254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:10.995254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:10.995302 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:10.995429 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:10.995429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:10.995429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:10.995525 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:10.995594 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:10.995735 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 07:09:10.995793 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260927 07:09:10.997172 6511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:10.997313 6511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:10.998493 6648 tablet_service.cc:1467] Processing CreateTablet for tablet 13ab48f9635d49efb6cae8c838f5be9f (DEFAULT_TABLE table=test-table [id=4934fd115e604aa49b591f4a5079417a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:10.998507 6788 tablet_service.cc:1467] Processing CreateTablet for tablet 13ab48f9635d49efb6cae8c838f5be9f (DEFAULT_TABLE table=test-table [id=4934fd115e604aa49b591f4a5079417a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:10.998528 6927 tablet_service.cc:1467] Processing CreateTablet for tablet 13ab48f9635d49efb6cae8c838f5be9f (DEFAULT_TABLE table=test-table [id=4934fd115e604aa49b591f4a5079417a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:10.998653 6648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13ab48f9635d49efb6cae8c838f5be9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:10.998682 6927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13ab48f9635d49efb6cae8c838f5be9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:10.998721 6788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13ab48f9635d49efb6cae8c838f5be9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:10.999802 7104 tablet_bootstrap.cc:492] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Bootstrap starting.
I20260927 07:09:11.000090 7106 tablet_bootstrap.cc:492] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: Bootstrap starting.
I20260927 07:09:11.000229 7104 tablet_bootstrap.cc:654] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:11.000468 7106 tablet_bootstrap.cc:654] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:11.000800 7105 tablet_bootstrap.cc:492] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: Bootstrap starting.
I20260927 07:09:11.000952 7104 tablet_bootstrap.cc:492] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: No bootstrap required, opened a new log
I20260927 07:09:11.001039 7104 ts_tablet_manager.cc:1397] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:11.001174 7105 tablet_bootstrap.cc:654] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:11.001196 7104 raft_consensus.cc:348] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.001274 7104 raft_consensus.cc:374] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:11.001298 7104 raft_consensus.cc:729] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b2f0c38dc14d4f9c491ee3d6cb1d82, State: Initialized, Role: FOLLOWER
I20260927 07:09:11.001360 7104 consensus_queue.cc:260] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.001545 7106 tablet_bootstrap.cc:492] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: No bootstrap required, opened a new log
I20260927 07:09:11.001564 7104 ts_tablet_manager.cc:1428] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:11.001598 7106 ts_tablet_manager.cc:1397] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:11.001724 7106 raft_consensus.cc:348] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.001796 7106 raft_consensus.cc:374] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:11.001816 7106 raft_consensus.cc:729] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a128f6769c414a8b8d87959fe7772459, State: Initialized, Role: FOLLOWER
I20260927 07:09:11.001876 7106 consensus_queue.cc:260] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.001972 7106 ts_tablet_manager.cc:1428] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:11.002169 7105 tablet_bootstrap.cc:492] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: No bootstrap required, opened a new log
I20260927 07:09:11.002219 7105 ts_tablet_manager.cc:1397] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:11.002338 7105 raft_consensus.cc:348] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.002426 7105 raft_consensus.cc:374] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:11.002456 7105 raft_consensus.cc:729] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8b9119e8524d17a58137b6db876084, State: Initialized, Role: FOLLOWER
I20260927 07:09:11.002514 7105 consensus_queue.cc:260] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.002712 7105 ts_tablet_manager.cc:1428] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:11.355466 7075 raft_consensus.cc:482] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:11.355571 7075 raft_consensus.cc:504] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.355721 7075 leader_election.cc:288] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741), 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125)
I20260927 07:09:11.355938 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ab48f9635d49efb6cae8c838f5be9f" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084" is_pre_election: true
I20260927 07:09:11.355935 6668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ab48f9635d49efb6cae8c838f5be9f" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a128f6769c414a8b8d87959fe7772459" is_pre_election: true
I20260927 07:09:11.356024 6808 raft_consensus.cc:2454] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 0.
I20260927 07:09:11.356040 6668 raft_consensus.cc:2454] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 0.
I20260927 07:09:11.356173 6881 leader_election.cc:302] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8b9119e8524d17a58137b6db876084, f2b2f0c38dc14d4f9c491ee3d6cb1d82; no voters:
I20260927 07:09:11.356292 7075 raft_consensus.cc:2781] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:11.356341 7075 raft_consensus.cc:482] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:11.356361 7075 raft_consensus.cc:3037] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:11.357028 7075 raft_consensus.cc:504] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.357142 7075 leader_election.cc:288] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 election: Requested vote from peers a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741), 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125)
I20260927 07:09:11.357286 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ab48f9635d49efb6cae8c838f5be9f" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084"
I20260927 07:09:11.357303 6668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ab48f9635d49efb6cae8c838f5be9f" candidate_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a128f6769c414a8b8d87959fe7772459"
I20260927 07:09:11.357358 6808 raft_consensus.cc:3037] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:11.357363 6668 raft_consensus.cc:3037] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:11.358047 6668 raft_consensus.cc:2454] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 1.
I20260927 07:09:11.358047 6808 raft_consensus.cc:2454] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b2f0c38dc14d4f9c491ee3d6cb1d82 in term 1.
I20260927 07:09:11.358207 6881 leader_election.cc:302] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8b9119e8524d17a58137b6db876084, f2b2f0c38dc14d4f9c491ee3d6cb1d82; no voters:
I20260927 07:09:11.358318 7075 raft_consensus.cc:2781] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:11.358362 7075 raft_consensus.cc:686] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 LEADER]: Becoming Leader. State: Replica: f2b2f0c38dc14d4f9c491ee3d6cb1d82, State: Running, Role: LEADER
I20260927 07:09:11.358403 7075 consensus_queue.cc:237] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.359001 6510 catalog_manager.cc:5495] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67). New cstate: current_term: 1 leader_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } health_report { overall_health: HEALTHY } } }
I20260927 07:09:11.445245 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:11.443074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.4.174.67:39611)
0927 07:09:11.443228 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:11.443229 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:11.443235 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:09:11.443253 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:11.443253 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:11.443263 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:11.443317 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:11.443467 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:11.443946 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:11.444673 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:11.444886 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:11.444943 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:11.445037 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:11.445064 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:11.445097 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:11.445133 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 07:09:11.445150 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260927 07:09:11.447155 6668 raft_consensus.cc:1261] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Refusing update from remote peer f2b2f0c38dc14d4f9c491ee3d6cb1d82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:11.447350 7052 consensus_queue.cc:1035] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:11.448956 6807 raft_consensus.cc:1261] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Refusing update from remote peer f2b2f0c38dc14d4f9c491ee3d6cb1d82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:11.449370 7080 consensus_queue.cc:1035] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:11.456496 6510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:11.456580 6510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:11.457104 6510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: Sending DeleteTablet for 3 replicas of tablet 13ab48f9635d49efb6cae8c838f5be9f
I20260927 07:09:11.457321 6648 tablet_service.cc:1514] Processing DeleteTablet for tablet 13ab48f9635d49efb6cae8c838f5be9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:11 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:51176
I20260927 07:09:11.457343 6788 tablet_service.cc:1514] Processing DeleteTablet for tablet 13ab48f9635d49efb6cae8c838f5be9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:11 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:59150
I20260927 07:09:11.457419 7094 tablet_replica.cc:330] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: stopping tablet replica
I20260927 07:09:11.457424 7093 tablet_replica.cc:330] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: stopping tablet replica
I20260927 07:09:11.457460 7094 raft_consensus.cc:2229] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:11.457494 7094 raft_consensus.cc:2258] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:11.457497 7093 raft_consensus.cc:2229] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:11.457530 7093 raft_consensus.cc:2258] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:11.457628 6928 tablet_service.cc:1514] Processing DeleteTablet for tablet 13ab48f9635d49efb6cae8c838f5be9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:11 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50612
I20260927 07:09:11.457719 7095 tablet_replica.cc:330] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: stopping tablet replica
I20260927 07:09:11.457738 7094 ts_tablet_manager.cc:1905] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:11.457765 7095 raft_consensus.cc:2229] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:11.457813 7095 raft_consensus.cc:2258] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:11.457816 7093 ts_tablet_manager.cc:1905] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:11.457988 6510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 07:09:11.458048 7095 ts_tablet_manager.cc:1905] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 07:09:11.458112 6510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:11.459277 7093 ts_tablet_manager.cc:1918] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:11.459290 7094 ts_tablet_manager.cc:1918] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:11.459333 7094 log.cc:1196] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/13ab48f9635d49efb6cae8c838f5be9f
I20260927 07:09:11.459333 7093 log.cc:1196] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/13ab48f9635d49efb6cae8c838f5be9f
I20260927 07:09:11.459408 7095 ts_tablet_manager.cc:1918] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:11.459447 7095 log.cc:1196] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/13ab48f9635d49efb6cae8c838f5be9f
I20260927 07:09:11.459537 7093 ts_tablet_manager.cc:1939] T 13ab48f9635d49efb6cae8c838f5be9f P a128f6769c414a8b8d87959fe7772459: Deleting consensus metadata
I20260927 07:09:11.459537 7094 ts_tablet_manager.cc:1939] T 13ab48f9635d49efb6cae8c838f5be9f P 3e8b9119e8524d17a58137b6db876084: Deleting consensus metadata
I20260927 07:09:11.459582 7095 ts_tablet_manager.cc:1939] T 13ab48f9635d49efb6cae8c838f5be9f P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting consensus metadata
I20260927 07:09:11.460027 6498 catalog_manager.cc:4841] TS a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741): tablet 13ab48f9635d49efb6cae8c838f5be9f (table test-table [id=4934fd115e604aa49b591f4a5079417a]) successfully deleted
I20260927 07:09:11.460069 6497 catalog_manager.cc:4841] TS f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611): tablet 13ab48f9635d49efb6cae8c838f5be9f (table test-table [id=4934fd115e604aa49b591f4a5079417a]) successfully deleted
I20260927 07:09:11.460124 6497 catalog_manager.cc:4841] TS 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125): tablet 13ab48f9635d49efb6cae8c838f5be9f (table test-table [id=4934fd115e604aa49b591f4a5079417a]) successfully deleted
I20260927 07:09:11.460784 6788 tablet_service.cc:1467] Processing CreateTablet for tablet 108da2e35dde4848b7302c94e465b433 (DEFAULT_TABLE table=test-table [id=77c32e6e87844e78972ff594a9f4b550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:11.460798 6648 tablet_service.cc:1467] Processing CreateTablet for tablet 108da2e35dde4848b7302c94e465b433 (DEFAULT_TABLE table=test-table [id=77c32e6e87844e78972ff594a9f4b550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:11.460887 6788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108da2e35dde4848b7302c94e465b433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:11.460887 6648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108da2e35dde4848b7302c94e465b433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:11.461953 7106 tablet_bootstrap.cc:492] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459: Bootstrap starting.
I20260927 07:09:11.461953 7105 tablet_bootstrap.cc:492] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: Bootstrap starting.
I20260927 07:09:11.461979 6928 tablet_service.cc:1467] Processing CreateTablet for tablet 108da2e35dde4848b7302c94e465b433 (DEFAULT_TABLE table=test-table [id=77c32e6e87844e78972ff594a9f4b550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:11.462070 6928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108da2e35dde4848b7302c94e465b433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:11.462292 7106 tablet_bootstrap.cc:654] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:11.462456 7105 tablet_bootstrap.cc:654] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:11.462829 7106 tablet_bootstrap.cc:492] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459: No bootstrap required, opened a new log
I20260927 07:09:11.462869 7106 ts_tablet_manager.cc:1397] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:11.462968 7106 raft_consensus.cc:348] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.463020 7104 tablet_bootstrap.cc:492] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Bootstrap starting.
I20260927 07:09:11.463032 7106 raft_consensus.cc:374] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:11.463052 7106 raft_consensus.cc:729] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a128f6769c414a8b8d87959fe7772459, State: Initialized, Role: FOLLOWER
I20260927 07:09:11.463084 7105 tablet_bootstrap.cc:492] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: No bootstrap required, opened a new log
I20260927 07:09:11.463090 7106 consensus_queue.cc:260] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.463120 7105 ts_tablet_manager.cc:1397] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:11.463191 7106 ts_tablet_manager.cc:1428] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:11.463271 7105 raft_consensus.cc:348] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.463321 7104 tablet_bootstrap.cc:654] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:11.463330 7105 raft_consensus.cc:374] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:11.463348 7105 raft_consensus.cc:729] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8b9119e8524d17a58137b6db876084, State: Initialized, Role: FOLLOWER
I20260927 07:09:11.463387 7105 consensus_queue.cc:260] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.463471 7105 ts_tablet_manager.cc:1428] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:11.463881 7104 tablet_bootstrap.cc:492] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: No bootstrap required, opened a new log
I20260927 07:09:11.463932 7104 ts_tablet_manager.cc:1397] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:11.464030 7104 raft_consensus.cc:348] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.464085 7104 raft_consensus.cc:374] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:11.464102 7104 raft_consensus.cc:729] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b2f0c38dc14d4f9c491ee3d6cb1d82, State: Initialized, Role: FOLLOWER
I20260927 07:09:11.464138 7104 consensus_queue.cc:260] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.464231 7104 ts_tablet_manager.cc:1428] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:11.765609 7085 raft_consensus.cc:482] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:11.765703 7085 raft_consensus.cc:504] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.765874 7085 leader_election.cc:288] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125), f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611)
I20260927 07:09:11.766110 6807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108da2e35dde4848b7302c94e465b433" candidate_uuid: "a128f6769c414a8b8d87959fe7772459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084" is_pre_election: true
I20260927 07:09:11.766144 6947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108da2e35dde4848b7302c94e465b433" candidate_uuid: "a128f6769c414a8b8d87959fe7772459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" is_pre_election: true
I20260927 07:09:11.766214 6807 raft_consensus.cc:2454] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a128f6769c414a8b8d87959fe7772459 in term 0.
I20260927 07:09:11.766225 6947 raft_consensus.cc:2454] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a128f6769c414a8b8d87959fe7772459 in term 0.
I20260927 07:09:11.766367 6601 leader_election.cc:302] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8b9119e8524d17a58137b6db876084, a128f6769c414a8b8d87959fe7772459; no voters:
I20260927 07:09:11.766458 7085 raft_consensus.cc:2781] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:11.766517 7085 raft_consensus.cc:482] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:11.766539 7085 raft_consensus.cc:3037] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:11.767258 7085 raft_consensus.cc:504] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.767362 7085 leader_election.cc:288] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125), f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611)
I20260927 07:09:11.767560 6807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108da2e35dde4848b7302c94e465b433" candidate_uuid: "a128f6769c414a8b8d87959fe7772459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8b9119e8524d17a58137b6db876084"
I20260927 07:09:11.767597 6947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108da2e35dde4848b7302c94e465b433" candidate_uuid: "a128f6769c414a8b8d87959fe7772459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82"
I20260927 07:09:11.767660 6807 raft_consensus.cc:3037] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:11.767673 6947 raft_consensus.cc:3037] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:11.768514 6947 raft_consensus.cc:2454] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a128f6769c414a8b8d87959fe7772459 in term 1.
I20260927 07:09:11.768514 6807 raft_consensus.cc:2454] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a128f6769c414a8b8d87959fe7772459 in term 1.
I20260927 07:09:11.768733 6601 leader_election.cc:302] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a128f6769c414a8b8d87959fe7772459, f2b2f0c38dc14d4f9c491ee3d6cb1d82; no voters:
I20260927 07:09:11.768827 7085 raft_consensus.cc:2781] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:11.768944 7085 raft_consensus.cc:686] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [term 1 LEADER]: Becoming Leader. State: Replica: a128f6769c414a8b8d87959fe7772459, State: Running, Role: LEADER
I20260927 07:09:11.768999 7085 consensus_queue.cc:237] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } }
I20260927 07:09:11.769522 6510 catalog_manager.cc:5495] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 reported cstate change: term changed from 0 to 1, leader changed from <none> to a128f6769c414a8b8d87959fe7772459 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "a128f6769c414a8b8d87959fe7772459" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a128f6769c414a8b8d87959fe7772459" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:11.818291 7123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:11.816153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.4.174.65:42741)
0927 07:09:11.816324 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:11.816326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:11.816335 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:11.816385 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:11.816386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:11.816397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:11.816451 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:11.816557 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:11.817024 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:11.817773 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:11.817955 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:11.817996 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:11.818085 (+ 89us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:11.818121 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:11.818154 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:11.818211 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 07:09:11.818226 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260927 07:09:11.820307 6947 raft_consensus.cc:1261] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Refusing update from remote peer a128f6769c414a8b8d87959fe7772459: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:11.820320 6807 raft_consensus.cc:1261] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Refusing update from remote peer a128f6769c414a8b8d87959fe7772459: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:11.820499 7085 consensus_queue.cc:1035] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8b9119e8524d17a58137b6db876084" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:11.820602 7121 consensus_queue.cc:1035] T 108da2e35dde4848b7302c94e465b433 P a128f6769c414a8b8d87959fe7772459 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b2f0c38dc14d4f9c491ee3d6cb1d82" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 07:09:11.824849 7077 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 07:09:13.340907 6508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:13.341010 6508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:13.341769 6508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac56a3ba9cf24989b56c9da254b0c6c4: Sending DeleteTablet for 3 replicas of tablet 108da2e35dde4848b7302c94e465b433
I20260927 07:09:13.342020 6648 tablet_service.cc:1514] Processing DeleteTablet for tablet 108da2e35dde4848b7302c94e465b433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:13 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:51176
I20260927 07:09:13.342034 6788 tablet_service.cc:1514] Processing DeleteTablet for tablet 108da2e35dde4848b7302c94e465b433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:13 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:59150
I20260927 07:09:13.342041 6928 tablet_service.cc:1514] Processing DeleteTablet for tablet 108da2e35dde4848b7302c94e465b433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:13 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50612
I20260927 07:09:13.342150 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 6583
I20260927 07:09:13.343655 7151 tablet_replica.cc:330] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: stopping tablet replica
I20260927 07:09:13.343720 7151 raft_consensus.cc:2229] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:13.343755 7151 raft_consensus.cc:2258] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:13.343971 7151 ts_tablet_manager.cc:1905] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:13.343967 7150 tablet_replica.cc:330] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: stopping tablet replica
I20260927 07:09:13.344019 7150 raft_consensus.cc:2229] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:13.344048 7150 raft_consensus.cc:2258] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:13.344269 7150 ts_tablet_manager.cc:1905] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:13.345444 7151 ts_tablet_manager.cc:1918] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:09:13.345500 7151 log.cc:1196] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/108da2e35dde4848b7302c94e465b433
I20260927 07:09:13.345721 7151 ts_tablet_manager.cc:1939] T 108da2e35dde4848b7302c94e465b433 P f2b2f0c38dc14d4f9c491ee3d6cb1d82: Deleting consensus metadata
I20260927 07:09:13.345733 7150 ts_tablet_manager.cc:1918] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:09:13.345779 7150 log.cc:1196] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/108da2e35dde4848b7302c94e465b433
I20260927 07:09:13.345978 7150 ts_tablet_manager.cc:1939] T 108da2e35dde4848b7302c94e465b433 P 3e8b9119e8524d17a58137b6db876084: Deleting consensus metadata
I20260927 07:09:13.346010 6497 catalog_manager.cc:4841] TS f2b2f0c38dc14d4f9c491ee3d6cb1d82 (127.4.174.67:39611): tablet 108da2e35dde4848b7302c94e465b433 (table test-table [id=77c32e6e87844e78972ff594a9f4b550]) successfully deleted
I20260927 07:09:13.346261 6497 catalog_manager.cc:4841] TS 3e8b9119e8524d17a58137b6db876084 (127.4.174.66:36125): tablet 108da2e35dde4848b7302c94e465b433 (table test-table [id=77c32e6e87844e78972ff594a9f4b550]) successfully deleted
W20260927 07:09:13.348364 6498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:42741 (error 108)
I20260927 07:09:13.348563 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 6723
I20260927 07:09:13.349314 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:13.349102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42741 (local address 127.0.0.1:51204)
0927 07:09:13.349223 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:13.349274 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:42741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
W20260927 07:09:13.349434 6498 catalog_manager.cc:4584] TS a128f6769c414a8b8d87959fe7772459 (127.4.174.65:42741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 108da2e35dde4848b7302c94e465b433: Network error: Client connection negotiation failed: client connection to 127.4.174.65:42741: connect: Connection refused (error 111)
I20260927 07:09:13.353497 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 6863
I20260927 07:09:13.358389 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 6479
2026-09-27T07:09:13Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6276 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:13 dist-test-slave-zw7p krb5kdc[7156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:13 dist-test-slave-zw7p krb5kdc[7156](info): set up 2 sockets
Sep 27 07:09:13 dist-test-slave-zw7p krb5kdc[7156](info): commencing operation
krb5kdc: starting...
W20260927 07:09:14.322096 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:14 dist-test-slave-zw7p krb5kdc[7156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:14.434777 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:38043
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:44635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:38043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:14.513674 7169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:14.513893 7169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:14.513926 7169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:14.515103 7169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:14.515149 7169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:14.515172 7169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:14.515192 7169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:14.515209 7169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:14.516667 7169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:44635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:38043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:38043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:14.516912 7169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:14.517105 7169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:14.522034 7174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:14.522199 7175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:14.522111 7177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:14.522065 7169 server_base.cc:1034] running on GCE node
I20260927 07:09:14.522504 7169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:14.522794 7169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:14.523931 7169 hybrid_clock.cc:648] HybridClock initialized: now 1790492954523918 us; error 28 us; skew 500 ppm
Sep 27 07:09:14 dist-test-slave-zw7p krb5kdc[7156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492954, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:14.526127 7169 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:14.526517 7169 webserver.cc:460] Webserver started at http://127.4.174.126:37185/ using document root <none> and password file <none>
I20260927 07:09:14.526752 7169 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:14.526800 7169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:14.526921 7169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:14.527932 7169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "9fc67850a4fd4770b1aedfa0cfe37b66"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "5ad872bf2410669b02ac4b337e997831"
server_key_iv: "a4f74a48376c9e63f13fb7fe6b821c43"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.528254 7169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "9fc67850a4fd4770b1aedfa0cfe37b66"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "5ad872bf2410669b02ac4b337e997831"
server_key_iv: "a4f74a48376c9e63f13fb7fe6b821c43"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.529479 7169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:14.530154 7184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:14.530297 7169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:14.530376 7169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "9fc67850a4fd4770b1aedfa0cfe37b66"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "5ad872bf2410669b02ac4b337e997831"
server_key_iv: "a4f74a48376c9e63f13fb7fe6b821c43"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.530441 7169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:14.546365 7169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:14.548241 7169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:14.548381 7169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:14.552044 7169 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:38043
I20260927 07:09:14.552083 7236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:38043 every 8 connection(s)
I20260927 07:09:14.552410 7169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:14.553285 7237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:14.555572 7237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66: Bootstrap starting.
I20260927 07:09:14.556106 7237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:14.556337 7237 log.cc:824] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:14.556929 7237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66: No bootstrap required, opened a new log
I20260927 07:09:14.558454 7237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc67850a4fd4770b1aedfa0cfe37b66" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38043 } }
I20260927 07:09:14.558624 7237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:14.558653 7237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc67850a4fd4770b1aedfa0cfe37b66, State: Initialized, Role: FOLLOWER
I20260927 07:09:14.558763 7237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc67850a4fd4770b1aedfa0cfe37b66" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38043 } }
I20260927 07:09:14.558812 7237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:14.558831 7237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:14.558862 7237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:14.559149 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 7169
I20260927 07:09:14.559229 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:14.559374 4793 external_mini_cluster.cc:1428] Setting key 70f258950e3a4cb12886611954b3521b
I20260927 07:09:14.559360 7237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc67850a4fd4770b1aedfa0cfe37b66" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38043 } }
I20260927 07:09:14.559441 7237 leader_election.cc:302] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fc67850a4fd4770b1aedfa0cfe37b66; no voters:
I20260927 07:09:14.559609 7237 leader_election.cc:288] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:14.559641 7240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:14.559866 7237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:14.559993 7240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [term 1 LEADER]: Becoming Leader. State: Replica: 9fc67850a4fd4770b1aedfa0cfe37b66, State: Running, Role: LEADER
I20260927 07:09:14.560144 7240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc67850a4fd4770b1aedfa0cfe37b66" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38043 } }
I20260927 07:09:14.560525 7241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fc67850a4fd4770b1aedfa0cfe37b66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc67850a4fd4770b1aedfa0cfe37b66" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38043 } } }
I20260927 07:09:14.560596 7241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:14.560714 7241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fc67850a4fd4770b1aedfa0cfe37b66. Latest consensus state: current_term: 1 leader_uuid: "9fc67850a4fd4770b1aedfa0cfe37b66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc67850a4fd4770b1aedfa0cfe37b66" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38043 } } }
I20260927 07:09:14.560761 7241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:14.560963 7251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:14.561528 7251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:14.563339 7251 catalog_manager.cc:1347] Generated new cluster ID: da8d508ff7e14b6f8c9c690bf8af30f9
I20260927 07:09:14.563390 7251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:09:14 dist-test-slave-zw7p krb5kdc[7156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:14.571156 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:14.560007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.4.174.126:38043)
0927 07:09:14.560280 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:14.560291 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:14.560325 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:09:14.560456 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:14.560464 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:14.560540 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:14.560697 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:14.561429 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.562703 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.563690 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.564287 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.564404 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:14.567075 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:14.567079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:14.567079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:14.567098 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:14.569560 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:14.569951 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:14.569953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:14.569954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:14.570051 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:14.570246 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:14.570247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:14.570247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:14.570524 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:14.570691 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:14.570795 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 07:09:14.570921 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":71,"threads_started":1}
I20260927 07:09:14.574752 7251 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:14.575445 7251 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:14.579203 7251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fc67850a4fd4770b1aedfa0cfe37b66: Generated new TSK 0
I20260927 07:09:14.579372 7251 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:14.607481 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38043
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:44635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:14.687454 7263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:14.687664 7263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:14.687685 7263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:14.688824 7263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:14.688858 7263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:14.688885 7263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:14.690263 7263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:44635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:14.690454 7263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:14.690653 7263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:14.695408 7268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:14.695437 7269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:14.695560 7263 server_base.cc:1034] running on GCE node
W20260927 07:09:14.695444 7271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:14.695827 7263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:14.696058 7263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:14.697219 7263 hybrid_clock.cc:648] HybridClock initialized: now 1790492954697199 us; error 29 us; skew 500 ppm
Sep 27 07:09:14 dist-test-slave-zw7p krb5kdc[7156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492954, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:14.699779 7263 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:14.700225 7263 webserver.cc:460] Webserver started at http://127.4.174.65:38759/ using document root <none> and password file <none>
I20260927 07:09:14.700443 7263 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:14.700491 7263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:14.700621 7263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:14.701733 7263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "4d5d4180ca274b53bc11c040bc3d99f8"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "d11fecb2819abadc692a9c35a3814365"
server_key_iv: "4ef3d62966c95ff9d796148f3cda2d58"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.702062 7263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "4d5d4180ca274b53bc11c040bc3d99f8"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "d11fecb2819abadc692a9c35a3814365"
server_key_iv: "4ef3d62966c95ff9d796148f3cda2d58"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.703364 7263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:14.704074 7278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:14.704231 7263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:14.704310 7263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "4d5d4180ca274b53bc11c040bc3d99f8"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "d11fecb2819abadc692a9c35a3814365"
server_key_iv: "4ef3d62966c95ff9d796148f3cda2d58"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.704367 7263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:14.715162 7263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:14.717164 7263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:14.717299 7263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:14.717516 7263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:14.717818 7263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:14.717851 7263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:14.717880 7263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:14.717903 7263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:14.723515 7263 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:43057
I20260927 07:09:14.723572 7391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:43057 every 8 connection(s)
I20260927 07:09:14.723879 7263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:14 dist-test-slave-zw7p krb5kdc[7156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492954, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:14.731829 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 7263
I20260927 07:09:14.731921 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:14.732121 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:14.724564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:38313 (local address 127.4.174.126:38043)
0927 07:09:14.724614 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:14.724616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:14.724799 (+ 183us) server_negotiation.cc:407] Connection header received
0927 07:09:14.725256 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:14.725257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:14.725270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:14.725322 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:14.726195 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.726717 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.727640 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.727830 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.727875 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:14.729701 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:14.729703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:14.729703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:14.729710 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:14.730980 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:14.731295 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:14.731296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:14.731296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:14.731343 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:14.731515 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:14.731515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:14.731515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:14.731610 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:14.731675 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:14.731979 (+ 304us) server_negotiation.cc:299] Negotiation successful
0927 07:09:14.732055 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:09:14.732122 4793 external_mini_cluster.cc:1428] Setting key fb35c698abb090f64300b61f89ab694f
I20260927 07:09:14.732323 7394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:14.724522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38043 (local address 127.4.174.65:38313)
0927 07:09:14.724683 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:14.724720 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:14.725178 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:14.725376 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:14.725380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:14.725928 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:14.726127 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:14.726132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.726785 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.726786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:14.727566 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:14.727568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.727884 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.727885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:14.727977 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:14.728384 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:14.728391 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:14.729625 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:14.731039 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:14.731043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:14.731045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:14.731242 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:14.731385 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:14.731386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:14.731386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:14.731474 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:14.731675 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:14.731680 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:14.731841 (+ 161us) client_negotiation.cc:769] Sending connection context
0927 07:09:14.731953 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 07:09:14.732112 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260927 07:09:14.732795 7392 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38043
I20260927 07:09:14.732926 7392 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:14.733125 7392 heartbeater.cc:511] Master 127.4.174.126:38043 requested a full tablet report, sending...
I20260927 07:09:14.733662 7201 ts_manager.cc:194] Registered new tserver with Master: 4d5d4180ca274b53bc11c040bc3d99f8 (127.4.174.65:43057)
I20260927 07:09:14.734320 7201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:38313
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:14.755242 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38043
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:44635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:14.835031 7397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:14.835237 7397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:14.835269 7397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:14.836401 7397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:14.836442 7397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:14.836484 7397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:14.837886 7397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:44635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:14.838114 7397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:14.838320 7397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:14.842955 7403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:14.843019 7405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:14.843055 7402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:14.843292 7397 server_base.cc:1034] running on GCE node
I20260927 07:09:14.843482 7397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:14.843695 7397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:14.844831 7397 hybrid_clock.cc:648] HybridClock initialized: now 1790492954844815 us; error 28 us; skew 500 ppm
Sep 27 07:09:14 dist-test-slave-zw7p krb5kdc[7156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492954, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:14.847235 7397 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:14.847632 7397 webserver.cc:460] Webserver started at http://127.4.174.66:42367/ using document root <none> and password file <none>
I20260927 07:09:14.847810 7397 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:14.847849 7397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:14.847944 7397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:14.848948 7397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "f394c9df45d0435d925f2125e4df7e8e"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "83e225f752fc47493312892b3f92add9"
server_key_iv: "bbd1e1883bf8e50bc1e11a2d27d20b99"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.849259 7397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "f394c9df45d0435d925f2125e4df7e8e"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "83e225f752fc47493312892b3f92add9"
server_key_iv: "bbd1e1883bf8e50bc1e11a2d27d20b99"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.850467 7397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:14.851172 7412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:14.851329 7397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:14.851411 7397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "f394c9df45d0435d925f2125e4df7e8e"
format_stamp: "Formatted at 2026-09-27 07:09:14 on dist-test-slave-zw7p"
server_key: "83e225f752fc47493312892b3f92add9"
server_key_iv: "bbd1e1883bf8e50bc1e11a2d27d20b99"
server_key_version: "encryptionkey@0"
I20260927 07:09:14.851467 7397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:14.881342 7397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:14.883206 7397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:14.883322 7397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:14.883512 7397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:14.883816 7397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:14.883847 7397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:14.883869 7397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:14.883893 7397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:14.889103 7397 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:36079
I20260927 07:09:14.889144 7525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:36079 every 8 connection(s)
I20260927 07:09:14.889479 7397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:09:14 dist-test-slave-zw7p krb5kdc[7156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492954, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:14.898164 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:14.890246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:52471 (local address 127.4.174.126:38043)
0927 07:09:14.890304 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:14.890305 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:14.890450 (+ 145us) server_negotiation.cc:407] Connection header received
0927 07:09:14.890958 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:14.890959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:14.890970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:14.891035 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:14.891965 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.892520 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.893508 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.893702 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.893758 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:14.895734 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:14.895736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:14.895736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:14.895743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:14.896949 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:14.897280 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:14.897281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:14.897281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:14.897337 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:14.897574 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:14.897575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:14.897576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:14.897698 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:14.897764 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:14.898018 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 07:09:14.898094 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 07:09:14.898303 7528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:14.890173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38043 (local address 127.4.174.66:52471)
0927 07:09:14.890322 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:14.890365 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:14.890871 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:14.891098 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:14.891102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:14.891671 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:14.891875 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:14.891880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.892593 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.892594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:14.893399 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:14.893401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:14.893768 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:14.893769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:14.893867 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:14.894301 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:14.894310 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:14.895654 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:14.897013 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:14.897016 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:14.897018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:14.897221 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:14.897384 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:14.897385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:14.897385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:14.897475 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:14.897773 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:14.897778 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:14.897899 (+ 121us) client_negotiation.cc:769] Sending connection context
0927 07:09:14.897982 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 07:09:14.898104 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260927 07:09:14.898808 7526 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38043
I20260927 07:09:14.898909 7526 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:14.899089 7526 heartbeater.cc:511] Master 127.4.174.126:38043 requested a full tablet report, sending...
I20260927 07:09:14.899444 7201 ts_manager.cc:194] Registered new tserver with Master: f394c9df45d0435d925f2125e4df7e8e (127.4.174.66:36079)
I20260927 07:09:14.899643 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 7397
I20260927 07:09:14.899724 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:14.899901 4793 external_mini_cluster.cc:1428] Setting key a9c80fdd78d66d631938a30115b887f3
I20260927 07:09:14.899955 7201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:52471
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:14.921370 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38043
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:44635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:14.999850 7531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:15.000066 7531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:15.000109 7531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:15.001259 7531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:15.001293 7531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:15.001317 7531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:15.002780 7531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:44635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:15.002980 7531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:15.003171 7531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:15.007932 7536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:15.008049 7531 server_base.cc:1034] running on GCE node
W20260927 07:09:15.007951 7539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:15.008139 7537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:15.008344 7531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:15.008548 7531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:15.009678 7531 hybrid_clock.cc:648] HybridClock initialized: now 1790492955009667 us; error 27 us; skew 500 ppm
Sep 27 07:09:15 dist-test-slave-zw7p krb5kdc[7156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492955, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:15.012109 7531 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:15.012506 7531 webserver.cc:460] Webserver started at http://127.4.174.67:43771/ using document root <none> and password file <none>
I20260927 07:09:15.012688 7531 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:15.012727 7531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:15.012816 7531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:15.013844 7531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "323211dadddc4dc99188ee841938dfa6"
format_stamp: "Formatted at 2026-09-27 07:09:15 on dist-test-slave-zw7p"
server_key: "6d4032cb4b42d237a7369e2937fcb82d"
server_key_iv: "9d7f164b0246d3c174fcf104b25e6c3d"
server_key_version: "encryptionkey@0"
I20260927 07:09:15.014147 7531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "323211dadddc4dc99188ee841938dfa6"
format_stamp: "Formatted at 2026-09-27 07:09:15 on dist-test-slave-zw7p"
server_key: "6d4032cb4b42d237a7369e2937fcb82d"
server_key_iv: "9d7f164b0246d3c174fcf104b25e6c3d"
server_key_version: "encryptionkey@0"
I20260927 07:09:15.015378 7531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:15.016031 7546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:15.016193 7531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:15.016275 7531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "323211dadddc4dc99188ee841938dfa6"
format_stamp: "Formatted at 2026-09-27 07:09:15 on dist-test-slave-zw7p"
server_key: "6d4032cb4b42d237a7369e2937fcb82d"
server_key_iv: "9d7f164b0246d3c174fcf104b25e6c3d"
server_key_version: "encryptionkey@0"
I20260927 07:09:15.016342 7531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:15.027369 7531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:15.029430 7531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:15.029577 7531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:15.029783 7531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:15.030089 7531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:15.030120 7531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:15.030148 7531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:15.030172 7531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:15.035730 7531 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:46527
I20260927 07:09:15.035768 7659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:46527 every 8 connection(s)
I20260927 07:09:15.036088 7531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:09:15 dist-test-slave-zw7p krb5kdc[7156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492955, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:15.044973 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:15.036747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:43127 (local address 127.4.174.126:38043)
0927 07:09:15.036789 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:15.036791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:15.036995 (+ 204us) server_negotiation.cc:407] Connection header received
0927 07:09:15.037460 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:15.037461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:15.037472 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:15.037502 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:15.038393 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:15.038930 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:15.039931 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:15.040134 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:15.040188 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:15.042331 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:15.042333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:15.042333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:15.042342 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:15.043617 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:15.044041 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:15.044042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:15.044043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:15.044095 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:15.044413 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:15.044413 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:15.044414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:15.044529 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:15.044574 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:15.044837 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 07:09:15.044904 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:09:15.045234 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:15.036734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38043 (local address 127.4.174.67:43127)
0927 07:09:15.036870 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:15.036906 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:15.037372 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:15.037576 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:15.037580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:15.038126 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:15.038325 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:15.038329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:15.039001 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:15.039002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:15.039850 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:15.039852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:15.040216 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:15.040216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:15.040318 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:15.040802 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:15.040811 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:15.042207 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:15.043718 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:15.043724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:15.043727 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:15.043970 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:15.044175 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:15.044177 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:15.044177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:15.044318 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:15.044584 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:15.044589 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:15.044720 (+ 131us) client_negotiation.cc:769] Sending connection context
0927 07:09:15.044819 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 07:09:15.044962 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260927 07:09:15.045760 7660 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38043
I20260927 07:09:15.045876 7660 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:15.046075 7660 heartbeater.cc:511] Master 127.4.174.126:38043 requested a full tablet report, sending...
I20260927 07:09:15.046128 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 7531
I20260927 07:09:15.046211 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:15.046411 4793 external_mini_cluster.cc:1428] Setting key 476a18e16168f81d8d1cb4031dd69207
I20260927 07:09:15.046473 7201 ts_manager.cc:194] Registered new tserver with Master: 323211dadddc4dc99188ee841938dfa6 (127.4.174.67:46527)
I20260927 07:09:15.046948 7201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:43127
I20260927 07:09:15.047302 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 07:09:15.058212 7244 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:15.057285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.4.174.126:38043)
0927 07:09:15.057343 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:15.057345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:15.057447 (+ 102us) server_negotiation.cc:407] Connection header received
0927 07:09:15.057506 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:15.057507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:15.057517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:15.057579 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:15.058137 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38108: BlockingRecv error: recv got EOF from 127.0.0.1:38108 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 07:09:15.059254 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 7263
I20260927 07:09:15.063459 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 7397
I20260927 07:09:15.067245 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 7531
I20260927 07:09:15.071180 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 7169
2026-09-27T07:09:15Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1711 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:15 dist-test-slave-zw7p krb5kdc[7672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:15 dist-test-slave-zw7p krb5kdc[7672](info): set up 2 sockets
Sep 27 07:09:15 dist-test-slave-zw7p krb5kdc[7672](info): commencing operation
krb5kdc: starting...
W20260927 07:09:16.018098 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:16 dist-test-slave-zw7p krb5kdc[7672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:16.130769 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:46545
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:38063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:46545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:16.209530 7685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:16.209729 7685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:16.209750 7685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:16.210860 7685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:16.210892 7685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:16.210904 7685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:16.210915 7685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:16.210932 7685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:16.212332 7685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:38063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:46545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:46545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:16.212529 7685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:16.212695 7685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:16.217192 7690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:16.217192 7693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:16.217206 7691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:16.217768 7685 server_base.cc:1034] running on GCE node
I20260927 07:09:16.217963 7685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:16.218197 7685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:16.219326 7685 hybrid_clock.cc:648] HybridClock initialized: now 1790492956219313 us; error 31 us; skew 500 ppm
Sep 27 07:09:16 dist-test-slave-zw7p krb5kdc[7672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492956, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:16.221668 7685 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:16.222051 7685 webserver.cc:460] Webserver started at http://127.4.174.126:42923/ using document root <none> and password file <none>
I20260927 07:09:16.222251 7685 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:16.222296 7685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:16.222398 7685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:16.223452 7685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "82fa1c584b6f4f6b85dac4311a70f519"
format_stamp: "Formatted at 2026-09-27 07:09:16 on dist-test-slave-zw7p"
server_key: "9923065874169c798b0e9e1899af25b8"
server_key_iv: "f782a8cb83e03d22155c358accf58b8d"
server_key_version: "encryptionkey@0"
I20260927 07:09:16.223799 7685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "82fa1c584b6f4f6b85dac4311a70f519"
format_stamp: "Formatted at 2026-09-27 07:09:16 on dist-test-slave-zw7p"
server_key: "9923065874169c798b0e9e1899af25b8"
server_key_iv: "f782a8cb83e03d22155c358accf58b8d"
server_key_version: "encryptionkey@0"
I20260927 07:09:16.225085 7685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:16.225692 7700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:16.225853 7685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:16.225931 7685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "82fa1c584b6f4f6b85dac4311a70f519"
format_stamp: "Formatted at 2026-09-27 07:09:16 on dist-test-slave-zw7p"
server_key: "9923065874169c798b0e9e1899af25b8"
server_key_iv: "f782a8cb83e03d22155c358accf58b8d"
server_key_version: "encryptionkey@0"
I20260927 07:09:16.225999 7685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:16.237390 7685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:16.239147 7685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:16.239283 7685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:16.242885 7685 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:46545
I20260927 07:09:16.242925 7752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:46545 every 8 connection(s)
I20260927 07:09:16.243247 7685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:16.244067 7753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:16.245211 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 7685
I20260927 07:09:16.245299 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:16.245481 4793 external_mini_cluster.cc:1428] Setting key b3092c725e3cb653a124b432b3850f92
I20260927 07:09:16.246730 7753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519: Bootstrap starting.
I20260927 07:09:16.247455 7753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:16.247725 7753 log.cc:824] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:16.248334 7753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519: No bootstrap required, opened a new log
I20260927 07:09:16.250540 7753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fa1c584b6f4f6b85dac4311a70f519" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46545 } }
I20260927 07:09:16.250739 7753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:16.250784 7753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fa1c584b6f4f6b85dac4311a70f519, State: Initialized, Role: FOLLOWER
I20260927 07:09:16.250919 7753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fa1c584b6f4f6b85dac4311a70f519" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46545 } }
I20260927 07:09:16.251003 7753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:16.251040 7753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:16.251089 7753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:16.251773 7753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fa1c584b6f4f6b85dac4311a70f519" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46545 } }
I20260927 07:09:16.251864 7753 leader_election.cc:302] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fa1c584b6f4f6b85dac4311a70f519; no voters:
I20260927 07:09:16.252014 7753 leader_election.cc:288] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:16.252081 7758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 1 FOLLOWER]: Leader election won for term 1
Sep 27 07:09:16 dist-test-slave-zw7p krb5kdc[7672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:16.252274 7758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [term 1 LEADER]: Becoming Leader. State: Replica: 82fa1c584b6f4f6b85dac4311a70f519, State: Running, Role: LEADER
I20260927 07:09:16.252277 7753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:16.252388 7758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fa1c584b6f4f6b85dac4311a70f519" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46545 } }
I20260927 07:09:16.253444 7760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fa1c584b6f4f6b85dac4311a70f519. Latest consensus state: current_term: 1 leader_uuid: "82fa1c584b6f4f6b85dac4311a70f519" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fa1c584b6f4f6b85dac4311a70f519" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46545 } } }
I20260927 07:09:16.253531 7760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:16.253942 7759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82fa1c584b6f4f6b85dac4311a70f519" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fa1c584b6f4f6b85dac4311a70f519" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46545 } } }
I20260927 07:09:16.254019 7759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:16.254195 7773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:16.254704 7773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:16.256240 7773 catalog_manager.cc:1347] Generated new cluster ID: 7e03b6973c3e4b7e8c4b18f3f5b5d6de
I20260927 07:09:16.256292 7773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:16.259033 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:16.245975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.4.174.126:46545)
0927 07:09:16.246143 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:16.246154 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:16.246179 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:09:16.246622 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:16.246631 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:16.246688 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:16.246813 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:16.247548 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:16.248521 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:16.249574 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:16.250002 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:16.250090 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:16.252838 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:16.252842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:16.252843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:16.252853 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:16.254956 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:16.255358 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:16.255361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:16.255361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:16.255474 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:16.255676 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:16.255677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:16.255678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:16.255982 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:16.256129 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:16.258692 (+ 2563us) server_negotiation.cc:299] Negotiation successful
0927 07:09:16.258839 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260927 07:09:16.259861 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:09:16.267333 7773 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:16.268050 7773 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:16.271579 7773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82fa1c584b6f4f6b85dac4311a70f519: Generated new TSK 0
I20260927 07:09:16.271754 7773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:09:16.277668 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:16.274321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.4.174.126:46545)
0927 07:09:16.274365 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:16.274366 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:16.274507 (+ 141us) server_negotiation.cc:407] Connection header received
0927 07:09:16.274601 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:16.274602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:16.274613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:16.274670 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:16.275059 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:16.275562 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:16.276271 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:16.276498 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:16.276553 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:16.277238 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:16.277239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:16.277240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:16.277411 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:16.277428 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:16.277434 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:16.277482 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:16.277565 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 07:09:16.277593 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:09:16.279836 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 7685
2026-09-27T07:09:16Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1207 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:16 dist-test-slave-zw7p krb5kdc[7786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:16 dist-test-slave-zw7p krb5kdc[7786](info): set up 2 sockets
Sep 27 07:09:16 dist-test-slave-zw7p krb5kdc[7786](info): commencing operation
krb5kdc: starting...
W20260927 07:09:17.226246 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:17.336721 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:43469
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:17.413941 7799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:17.414176 7799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:17.414212 7799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:17.415406 7799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:17.415445 7799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:17.415463 7799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:17.415480 7799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:17.415494 7799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:17.416975 7799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:43469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:17.417213 7799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:17.417419 7799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:17.422057 7805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:17.422053 7807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:17.422214 7799 server_base.cc:1034] running on GCE node
W20260927 07:09:17.422053 7804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:17.422524 7799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:17.422830 7799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:17.423971 7799 hybrid_clock.cc:648] HybridClock initialized: now 1790492957423957 us; error 27 us; skew 500 ppm
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:17.426431 7799 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:17.426915 7799 webserver.cc:460] Webserver started at http://127.4.174.126:45827/ using document root <none> and password file <none>
I20260927 07:09:17.427135 7799 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:17.427183 7799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:17.427307 7799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:17.428331 7799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "f70d28f18fd14a7a8c5432d303e5c73f"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "94cc00ce92036e13e4138aca15b3b3ce"
server_key_iv: "2380fe07b25b6ce0aca704bd586dd6ea"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.428663 7799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "f70d28f18fd14a7a8c5432d303e5c73f"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "94cc00ce92036e13e4138aca15b3b3ce"
server_key_iv: "2380fe07b25b6ce0aca704bd586dd6ea"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.429889 7799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:17.430639 7814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.430794 7799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:17.430876 7799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "f70d28f18fd14a7a8c5432d303e5c73f"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "94cc00ce92036e13e4138aca15b3b3ce"
server_key_iv: "2380fe07b25b6ce0aca704bd586dd6ea"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.430943 7799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:17.440783 7799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:17.442519 7799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:17.442698 7799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:17.446446 7799 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:43469
I20260927 07:09:17.446491 7866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:43469 every 8 connection(s)
I20260927 07:09:17.446844 7799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:17.447791 7867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:17.450052 7867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f: Bootstrap starting.
I20260927 07:09:17.450665 7867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:17.450734 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 7799
I20260927 07:09:17.450834 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:17.450955 7867 log.cc:824] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:17.450989 4793 external_mini_cluster.cc:1428] Setting key bee62ae4b8294439ce39a0e03f9999e4
I20260927 07:09:17.451669 7867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f: No bootstrap required, opened a new log
I20260927 07:09:17.453647 7867 raft_consensus.cc:348] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70d28f18fd14a7a8c5432d303e5c73f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43469 } }
I20260927 07:09:17.453785 7867 raft_consensus.cc:374] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:17.453816 7867 raft_consensus.cc:729] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70d28f18fd14a7a8c5432d303e5c73f, State: Initialized, Role: FOLLOWER
I20260927 07:09:17.453958 7867 consensus_queue.cc:260] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70d28f18fd14a7a8c5432d303e5c73f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43469 } }
I20260927 07:09:17.454047 7867 raft_consensus.cc:388] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:17.454079 7867 raft_consensus.cc:482] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:17.454125 7867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:17.454670 7867 raft_consensus.cc:504] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70d28f18fd14a7a8c5432d303e5c73f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43469 } }
I20260927 07:09:17.454766 7867 leader_election.cc:302] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f70d28f18fd14a7a8c5432d303e5c73f; no voters:
I20260927 07:09:17.454914 7867 leader_election.cc:288] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:17.454947 7872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:17.455113 7872 raft_consensus.cc:686] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [term 1 LEADER]: Becoming Leader. State: Replica: f70d28f18fd14a7a8c5432d303e5c73f, State: Running, Role: LEADER
I20260927 07:09:17.455183 7867 sys_catalog.cc:564] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:17.455277 7872 consensus_queue.cc:237] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70d28f18fd14a7a8c5432d303e5c73f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43469 } }
I20260927 07:09:17.455605 7872 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f70d28f18fd14a7a8c5432d303e5c73f. Latest consensus state: current_term: 1 leader_uuid: "f70d28f18fd14a7a8c5432d303e5c73f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70d28f18fd14a7a8c5432d303e5c73f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43469 } } }
I20260927 07:09:17.455741 7872 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:17.455889 7874 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f70d28f18fd14a7a8c5432d303e5c73f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70d28f18fd14a7a8c5432d303e5c73f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43469 } } }
I20260927 07:09:17.455969 7874 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:17.456188 7881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:17.456678 7881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:17.458426 7881 catalog_manager.cc:1347] Generated new cluster ID: 05d88dc1508e46f0855330ccaea0951b
I20260927 07:09:17.458518 7881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:17.461544 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:17.451458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.4.174.126:43469)
0927 07:09:17.451623 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:17.451633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:17.451661 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:09:17.452122 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:17.452129 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:17.452179 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:17.452303 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:17.452902 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.453772 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.454599 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.455117 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.455220 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:17.457989 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:17.457994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:17.457994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:17.458005 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:17.459919 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:17.460366 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:17.460368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:17.460369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:17.460467 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:17.460663 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:17.460664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:17.460664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:17.460945 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:17.461101 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:17.461222 (+ 121us) server_negotiation.cc:299] Negotiation successful
0927 07:09:17.461350 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260927 07:09:17.477294 7881 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:17.478060 7881 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:17.483184 7881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f70d28f18fd14a7a8c5432d303e5c73f: Generated new TSK 0
I20260927 07:09:17.483387 7881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:17.513814 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:43469
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:17.591645 7893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:17.591835 7893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:17.591854 7893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:17.592954 7893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:17.592994 7893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:17.593021 7893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:17.594425 7893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:17.594676 7893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:17.594888 7893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:17.599603 7899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:17.599620 7893 server_base.cc:1034] running on GCE node
W20260927 07:09:17.599596 7901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:17.599596 7898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:17.600081 7893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:17.600303 7893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:17.601442 7893 hybrid_clock.cc:648] HybridClock initialized: now 1790492957601431 us; error 24 us; skew 500 ppm
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:17.603798 7893 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:17.604200 7893 webserver.cc:460] Webserver started at http://127.4.174.65:39955/ using document root <none> and password file <none>
I20260927 07:09:17.604406 7893 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:17.604453 7893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:17.604569 7893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:17.605552 7893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "844f9470c8a845f395eed337f8ebf25c"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "797c9f9c81562798ec71d7c06cbb4207"
server_key_iv: "9e6489e9f141fa67b69be869d358e270"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.605860 7893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "844f9470c8a845f395eed337f8ebf25c"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "797c9f9c81562798ec71d7c06cbb4207"
server_key_iv: "9e6489e9f141fa67b69be869d358e270"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.607123 7893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:17.607921 7908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.608114 7893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:17.608206 7893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "844f9470c8a845f395eed337f8ebf25c"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "797c9f9c81562798ec71d7c06cbb4207"
server_key_iv: "9e6489e9f141fa67b69be869d358e270"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.608266 7893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:17.650978 7893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:17.653124 7893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:17.653262 7893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:17.653481 7893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:17.653776 7893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:17.653810 7893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.653841 7893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:17.653861 7893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.659260 7893 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:33545
I20260927 07:09:17.659317 8021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:33545 every 8 connection(s)
I20260927 07:09:17.659613 7893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:17.667887 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:17.660316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:38559 (local address 127.4.174.126:43469)
0927 07:09:17.660367 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:17.660369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:17.660573 (+ 204us) server_negotiation.cc:407] Connection header received
0927 07:09:17.661039 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:17.661040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:17.661049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:17.661099 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:17.661964 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.662454 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.663457 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.663650 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.663695 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:17.665688 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:17.665689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:17.665690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:17.665697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:17.666773 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:17.667114 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:17.667115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:17.667115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:17.667162 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:17.667355 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:17.667356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:17.667356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:17.667452 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:17.667498 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:17.667741 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 07:09:17.667819 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 07:09:17.667999 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:17.660293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:43469 (local address 127.4.174.65:38559)
0927 07:09:17.660454 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:17.660496 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:17.660961 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:17.661160 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:17.661163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:17.661701 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:17.661898 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:17.661903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.662523 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.662524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:17.663396 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:17.663399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.663733 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.663734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:17.663855 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:17.664251 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:17.664258 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:17.665587 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:17.666836 (+ 1249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:17.666841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:17.666843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:17.667060 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:17.667204 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:17.667205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:17.667205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:17.667295 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:17.667494 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:17.667497 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:17.667608 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 07:09:17.667705 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 07:09:17.667822 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260927 07:09:17.668522 8022 heartbeater.cc:347] Connected to a master server at 127.4.174.126:43469
I20260927 07:09:17.668535 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 7893
I20260927 07:09:17.668606 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:17.668610 8022 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:17.668763 8022 heartbeater.cc:511] Master 127.4.174.126:43469 requested a full tablet report, sending...
I20260927 07:09:17.668783 4793 external_mini_cluster.cc:1428] Setting key 5356b5b6ab7c0db2c65bfdea4691682d
I20260927 07:09:17.669236 7831 ts_manager.cc:194] Registered new tserver with Master: 844f9470c8a845f395eed337f8ebf25c (127.4.174.65:33545)
I20260927 07:09:17.669864 7831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:38559
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:17.691728 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:43469
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:17.769567 8027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:17.769774 8027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:17.769797 8027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:17.770951 8027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:17.770992 8027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:17.771018 8027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:17.772395 8027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:17.772583 8027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:17.772768 8027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:17.778060 8032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:17.778059 8035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:17.778259 8033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:17.778626 8027 server_base.cc:1034] running on GCE node
I20260927 07:09:17.778817 8027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:17.779045 8027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:17.780187 8027 hybrid_clock.cc:648] HybridClock initialized: now 1790492957780169 us; error 29 us; skew 500 ppm
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:17.782505 8027 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:17.782943 8027 webserver.cc:460] Webserver started at http://127.4.174.66:45817/ using document root <none> and password file <none>
I20260927 07:09:17.783134 8027 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:17.783174 8027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:17.783262 8027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:17.784281 8027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "34742371bc5549988499fba28465b19a"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "97a2f835339ade328910ea9e02534e5f"
server_key_iv: "3414742e40936464bc6d13a683129f98"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.784578 8027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "34742371bc5549988499fba28465b19a"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "97a2f835339ade328910ea9e02534e5f"
server_key_iv: "3414742e40936464bc6d13a683129f98"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.785766 8027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:17.786393 8042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.786536 8027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:17.786644 8027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "34742371bc5549988499fba28465b19a"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "97a2f835339ade328910ea9e02534e5f"
server_key_iv: "3414742e40936464bc6d13a683129f98"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.786700 8027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:17.798166 8027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:17.800168 8027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:17.800275 8027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:17.800453 8027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:17.800704 8027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:17.800733 8027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.800755 8027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:17.800770 8027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.806102 8027 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:42979
I20260927 07:09:17.806167 8155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:42979 every 8 connection(s)
I20260927 07:09:17.806440 8027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:17.814788 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:17.807191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:41741 (local address 127.4.174.126:43469)
0927 07:09:17.807256 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:17.807258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:17.807455 (+ 197us) server_negotiation.cc:407] Connection header received
0927 07:09:17.807910 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:17.807911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:17.807920 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:17.807972 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:17.808866 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.809355 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.810288 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.810463 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.810519 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:17.812535 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:17.812537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:17.812537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:17.812545 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:17.813746 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:17.814062 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:17.814063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:17.814063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:17.814110 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:17.814279 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:17.814280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:17.814280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:17.814376 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:17.814418 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:17.814658 (+ 240us) server_negotiation.cc:299] Negotiation successful
0927 07:09:17.814719 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:17.814949 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:17.807171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:43469 (local address 127.4.174.66:41741)
0927 07:09:17.807335 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:17.807373 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:17.807836 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:17.808044 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:17.808048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:17.808598 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:17.808795 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:17.808800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.809417 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.809418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:17.810229 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:17.810231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.810521 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.810521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:17.810678 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:17.811141 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:17.811150 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:17.812439 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:17.813805 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:17.813808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:17.813809 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:17.814011 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:17.814155 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:17.814155 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:17.814156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:17.814241 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:17.814415 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:17.814418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:17.814530 (+ 112us) client_negotiation.cc:769] Sending connection context
0927 07:09:17.814641 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 07:09:17.814759 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260927 07:09:17.815361 8156 heartbeater.cc:347] Connected to a master server at 127.4.174.126:43469
I20260927 07:09:17.815444 8156 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:17.815589 8156 heartbeater.cc:511] Master 127.4.174.126:43469 requested a full tablet report, sending...
I20260927 07:09:17.815948 7831 ts_manager.cc:194] Registered new tserver with Master: 34742371bc5549988499fba28465b19a (127.4.174.66:42979)
I20260927 07:09:17.815944 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 8027
I20260927 07:09:17.816049 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:17.816256 4793 external_mini_cluster.cc:1428] Setting key bd88d21f19b0f418a33ac0b428796475
I20260927 07:09:17.816402 7831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:41741
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:17.838819 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:43469
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:17.917424 8161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:17.917650 8161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:17.917673 8161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:17.918810 8161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:17.918846 8161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:17.918871 8161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:17.920298 8161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:17.920506 8161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:17.920691 8161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:17.925493 8166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:17.925503 8167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:17.925561 8161 server_base.cc:1034] running on GCE node
W20260927 07:09:17.925504 8169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:17.925861 8161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:17.926110 8161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:17.927256 8161 hybrid_clock.cc:648] HybridClock initialized: now 1790492957927242 us; error 24 us; skew 500 ppm
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:17.929563 8161 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:17.929968 8161 webserver.cc:460] Webserver started at http://127.4.174.67:41165/ using document root <none> and password file <none>
I20260927 07:09:17.930186 8161 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:17.930233 8161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:17.930351 8161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:17.931442 8161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "e17a176e040a41b38f111c70106932df"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "3f30bd15bdfecd6da4c899b5f47e7801"
server_key_iv: "0e1ce8bcc372aba1879cf754274c265a"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.931790 8161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "e17a176e040a41b38f111c70106932df"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "3f30bd15bdfecd6da4c899b5f47e7801"
server_key_iv: "0e1ce8bcc372aba1879cf754274c265a"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.933007 8161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:17.933657 8176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.933804 8161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:17.933882 8161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "e17a176e040a41b38f111c70106932df"
format_stamp: "Formatted at 2026-09-27 07:09:17 on dist-test-slave-zw7p"
server_key: "3f30bd15bdfecd6da4c899b5f47e7801"
server_key_iv: "0e1ce8bcc372aba1879cf754274c265a"
server_key_version: "encryptionkey@0"
I20260927 07:09:17.933965 8161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:17.975032 8161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:17.976904 8161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:17.977047 8161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:17.977268 8161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:17.977573 8161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:17.977605 8161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.977633 8161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:17.977658 8161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:17.983340 8161 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:35897
I20260927 07:09:17.983412 8289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:35897 every 8 connection(s)
I20260927 07:09:17.983704 8161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:09:17 dist-test-slave-zw7p krb5kdc[7786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492957, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:17.992374 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:17.984355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:60689 (local address 127.4.174.126:43469)
0927 07:09:17.984407 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:17.984409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:17.984590 (+ 181us) server_negotiation.cc:407] Connection header received
0927 07:09:17.985096 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:17.985097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:17.985109 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:17.985165 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:17.986128 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.986907 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.987859 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.988062 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.988118 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:17.990018 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:17.990019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:17.990020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:17.990027 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:17.991187 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:17.991533 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:17.991534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:17.991535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:17.991587 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:17.991790 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:17.991791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:17.991791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:17.991897 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:17.991953 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:17.992209 (+ 256us) server_negotiation.cc:299] Negotiation successful
0927 07:09:17.992286 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 07:09:17.992534 8292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:17.984339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:43469 (local address 127.4.174.67:60689)
0927 07:09:17.984476 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:17.984512 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:17.985004 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:17.985237 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:17.985240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:17.985782 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:17.986008 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:17.986015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.986984 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.986985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:17.987777 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:17.987779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:17.988128 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:17.988129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:17.988239 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:17.988643 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:17.988650 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:17.989923 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:17.991256 (+ 1333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:17.991262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:17.991265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:17.991469 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:17.991638 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:17.991638 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:17.991639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:17.991737 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:17.991959 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:17.991965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:17.992095 (+ 130us) client_negotiation.cc:769] Sending connection context
0927 07:09:17.992180 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:09:17.992323 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260927 07:09:17.992949 8290 heartbeater.cc:347] Connected to a master server at 127.4.174.126:43469
I20260927 07:09:17.993046 8290 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:17.993208 8290 heartbeater.cc:511] Master 127.4.174.126:43469 requested a full tablet report, sending...
I20260927 07:09:17.993392 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 8161
I20260927 07:09:17.993466 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:17.993554 7831 ts_manager.cc:194] Registered new tserver with Master: e17a176e040a41b38f111c70106932df (127.4.174.67:35897)
I20260927 07:09:17.993639 4793 external_mini_cluster.cc:1428] Setting key 151a973f97d4e7478ee2b39fde54522b
I20260927 07:09:17.994014 7831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:60689
I20260927 07:09:17.994503 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:09:18 dist-test-slave-zw7p krb5kdc[7786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492958, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 27 07:09:18 dist-test-slave-zw7p krb5kdc[7786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492958, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:18.015221 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:18.008894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.4.174.126:43469)
0927 07:09:18.008969 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:18.008972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:18.009046 (+ 74us) server_negotiation.cc:407] Connection header received
0927 07:09:18.009116 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:18.009117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:18.009127 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:18.009169 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:18.009702 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:18.010210 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:18.010983 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:18.011189 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:18.011252 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:18.013221 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:18.013222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:18.013223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:18.013230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:18.014300 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:18.014620 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:18.014621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:18.014621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:18.014668 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:18.014806 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:18.014806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:18.014807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:18.014941 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:18.015024 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:18.015088 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 07:09:18.015154 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 07:09:18.015430 7831 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44646
I20260927 07:09:18.015864 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 7893
I20260927 07:09:18.020418 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 8027
I20260927 07:09:18.024207 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 8161
I20260927 07:09:18.027755 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 7799
2026-09-27T07:09:18Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1749 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:18 dist-test-slave-zw7p krb5kdc[8302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:18 dist-test-slave-zw7p krb5kdc[8302](info): set up 2 sockets
Sep 27 07:09:18 dist-test-slave-zw7p krb5kdc[8302](info): commencing operation
krb5kdc: starting...
W20260927 07:09:18.978098 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:19.087312 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40071
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:19.164846 8315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:19.165056 8315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:19.165087 8315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:19.166230 8315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:19.166270 8315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:19.166287 8315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:19.166303 8315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:19.166318 8315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:19.167819 8315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:19.168062 8315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:19.168264 8315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:19.172814 8320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:19.172796 8323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:19.172794 8321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:19.173487 8315 server_base.cc:1034] running on GCE node
I20260927 07:09:19.173699 8315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:19.173950 8315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:19.175091 8315 hybrid_clock.cc:648] HybridClock initialized: now 1790492959175077 us; error 30 us; skew 500 ppm
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:19.182894 8315 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:19.183367 8315 webserver.cc:460] Webserver started at http://127.4.174.126:33561/ using document root <none> and password file <none>
I20260927 07:09:19.183591 8315 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:19.183641 8315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:19.183764 8315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:19.184809 8315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "f5c81da658584bbd9319c96cc765fade"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "a5b296e85f260ff4f5ebc72ef6d35925"
server_key_iv: "19959708f3689ba43ddbc639961c5f9d"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.185151 8315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "f5c81da658584bbd9319c96cc765fade"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "a5b296e85f260ff4f5ebc72ef6d35925"
server_key_iv: "19959708f3689ba43ddbc639961c5f9d"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.186400 8315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 07:09:19.187150 8330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.187322 8315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.187405 8315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "f5c81da658584bbd9319c96cc765fade"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "a5b296e85f260ff4f5ebc72ef6d35925"
server_key_iv: "19959708f3689ba43ddbc639961c5f9d"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.187470 8315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:19.196563 8315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:19.198403 8315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:19.198540 8315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:19.202324 8315 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40071
I20260927 07:09:19.202358 8382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40071 every 8 connection(s)
I20260927 07:09:19.202847 8315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:19.203565 8383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:19.205790 8383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade: Bootstrap starting.
I20260927 07:09:19.206346 8383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:19.206609 8383 log.cc:824] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:19.207242 8383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade: No bootstrap required, opened a new log
I20260927 07:09:19.208786 8383 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c81da658584bbd9319c96cc765fade" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40071 } }
I20260927 07:09:19.208928 8383 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:19.208958 8383 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c81da658584bbd9319c96cc765fade, State: Initialized, Role: FOLLOWER
I20260927 07:09:19.209066 8383 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c81da658584bbd9319c96cc765fade" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40071 } }
I20260927 07:09:19.209115 8383 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:19.209141 8383 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:19.209170 8383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:19.209995 8383 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c81da658584bbd9319c96cc765fade" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40071 } }
I20260927 07:09:19.210083 8383 leader_election.cc:302] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5c81da658584bbd9319c96cc765fade; no voters:
I20260927 07:09:19.210210 8383 leader_election.cc:288] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:19.210258 8386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:19.210389 8386 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [term 1 LEADER]: Becoming Leader. State: Replica: f5c81da658584bbd9319c96cc765fade, State: Running, Role: LEADER
I20260927 07:09:19.210464 8383 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:19.210520 8386 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c81da658584bbd9319c96cc765fade" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40071 } }
I20260927 07:09:19.210887 8387 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5c81da658584bbd9319c96cc765fade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c81da658584bbd9319c96cc765fade" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40071 } } }
I20260927 07:09:19.210893 8388 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5c81da658584bbd9319c96cc765fade. Latest consensus state: current_term: 1 leader_uuid: "f5c81da658584bbd9319c96cc765fade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c81da658584bbd9319c96cc765fade" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40071 } } }
I20260927 07:09:19.211068 8388 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:19.211061 8387 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:19.211431 8395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:19.211603 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 8315
I20260927 07:09:19.211715 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:19.211905 4793 external_mini_cluster.cc:1428] Setting key 8f98bcc2750c25dedfc1ed04dcf9730f
I20260927 07:09:19.211956 8395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:19.213677 8395 catalog_manager.cc:1347] Generated new cluster ID: 7899b1d8fd1b492192e0f19f41d16566
I20260927 07:09:19.213719 8395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:19.223989 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.212312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.4.174.126:40071)
0927 07:09:19.212593 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:19.212604 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:19.212637 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:09:19.212752 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:19.212759 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:19.212828 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:19.212968 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:19.214695 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.216119 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.217129 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.217761 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.217902 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.220570 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:19.220575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:19.220575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:19.220588 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:19.222384 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.222787 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.222789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.222790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.222892 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.223070 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.223070 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.223071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.223376 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:19.223516 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:19.223623 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 07:09:19.223753 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":74,"threads_started":1}
I20260927 07:09:19.237609 8395 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:19.238134 8395 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:19.246263 8395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5c81da658584bbd9319c96cc765fade: Generated new TSK 0
I20260927 07:09:19.246470 8395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:19.269558 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40071
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:19.347656 8409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:19.347877 8409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:19.347910 8409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:19.349092 8409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:19.349133 8409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:19.349170 8409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:19.350786 8409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:19.351054 8409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:19.351277 8409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:19.356112 8414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:19.356139 8409 server_base.cc:1034] running on GCE node
W20260927 07:09:19.356216 8417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:19.356163 8415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:19.356549 8409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:19.356778 8409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:19.357918 8409 hybrid_clock.cc:648] HybridClock initialized: now 1790492959357902 us; error 26 us; skew 500 ppm
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:19.360332 8409 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:19.360774 8409 webserver.cc:460] Webserver started at http://127.4.174.65:37285/ using document root <none> and password file <none>
I20260927 07:09:19.360989 8409 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:19.361037 8409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:19.361160 8409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:19.362171 8409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "534944e78b5f42bf98f2dce41e11f235"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "4dce51286ee4613298045b3f3198220d"
server_key_iv: "99df5daa0657c0f5484d74bddf942d56"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.362495 8409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "534944e78b5f42bf98f2dce41e11f235"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "4dce51286ee4613298045b3f3198220d"
server_key_iv: "99df5daa0657c0f5484d74bddf942d56"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.363709 8409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:19.364365 8424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.364508 8409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:19.364588 8409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "534944e78b5f42bf98f2dce41e11f235"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "4dce51286ee4613298045b3f3198220d"
server_key_iv: "99df5daa0657c0f5484d74bddf942d56"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.364647 8409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:19.392515 8409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:19.394234 8409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:19.394371 8409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:19.394618 8409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:19.394951 8409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:19.394991 8409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.395021 8409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:19.395042 8409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.400382 8409 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:46867
I20260927 07:09:19.400439 8537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:46867 every 8 connection(s)
I20260927 07:09:19.400756 8409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:19.404316 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 8409
I20260927 07:09:19.404407 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:19.404583 4793 external_mini_cluster.cc:1428] Setting key 67e47b0244ce4b18b22e71151bb20827
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:19.409973 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.401439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:39141 (local address 127.4.174.126:40071)
0927 07:09:19.401489 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:19.401491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:19.401684 (+ 193us) server_negotiation.cc:407] Connection header received
0927 07:09:19.402643 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:19.402645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:19.402663 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:19.402714 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:19.403326 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.403829 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.404900 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.405125 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.405186 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.407279 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:19.407281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:19.407281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:19.407288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:19.408590 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.409041 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.409041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.409042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.409091 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.409347 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.409348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.409348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.409447 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:19.409493 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:19.409821 (+ 328us) server_negotiation.cc:299] Negotiation successful
0927 07:09:19.409896 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:19.410244 8540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.401404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40071 (local address 127.4.174.65:39141)
0927 07:09:19.401556 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:19.401595 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:19.402515 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:19.402786 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:19.402790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:19.403048 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:19.403237 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:19.403242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.403911 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.403912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:19.404835 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:19.404838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.405207 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.405208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:19.405333 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.405821 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:19.405829 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:19.407189 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:19.408665 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:19.408672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:19.408675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:19.408968 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:19.409147 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:19.409147 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:19.409148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:19.409291 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:19.409503 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:19.409509 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:19.409686 (+ 177us) client_negotiation.cc:769] Sending connection context
0927 07:09:19.409810 (+ 124us) client_negotiation.cc:240] Negotiation successful
0927 07:09:19.409986 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260927 07:09:19.410835 8538 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40071
I20260927 07:09:19.410933 8538 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:19.411134 8538 heartbeater.cc:511] Master 127.4.174.126:40071 requested a full tablet report, sending...
I20260927 07:09:19.411708 8347 ts_manager.cc:194] Registered new tserver with Master: 534944e78b5f42bf98f2dce41e11f235 (127.4.174.65:46867)
I20260927 07:09:19.412361 8347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:39141
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:19.429457 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40071
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:19.509356 8543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:19.509573 8543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:19.509598 8543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:19.510798 8543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:19.510841 8543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:19.510867 8543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:19.512502 8543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:19.512693 8543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:19.512880 8543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:19.517659 8548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:19.517660 8551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:19.517660 8549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:19.517918 8543 server_base.cc:1034] running on GCE node
I20260927 07:09:19.518113 8543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:19.518344 8543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:19.519555 8543 hybrid_clock.cc:648] HybridClock initialized: now 1790492959519541 us; error 30 us; skew 500 ppm
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:19.521929 8543 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:19.522341 8543 webserver.cc:460] Webserver started at http://127.4.174.66:33773/ using document root <none> and password file <none>
I20260927 07:09:19.522516 8543 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:19.522622 8543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:19.522729 8543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:19.523741 8543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "ce111a5177c645eb877e2f4748e07bed"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "3e05d0361513f37698fdfe25b672b1d1"
server_key_iv: "e758ba6e0814c73cd7533747d84b1e4b"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.524031 8543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "ce111a5177c645eb877e2f4748e07bed"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "3e05d0361513f37698fdfe25b672b1d1"
server_key_iv: "e758ba6e0814c73cd7533747d84b1e4b"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.525204 8543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:19.525846 8558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.525997 8543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:19.526077 8543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "ce111a5177c645eb877e2f4748e07bed"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "3e05d0361513f37698fdfe25b672b1d1"
server_key_iv: "e758ba6e0814c73cd7533747d84b1e4b"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.526129 8543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:19.538933 8543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:19.541132 8543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:19.541260 8543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:19.541476 8543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:19.541759 8543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:19.541788 8543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.541811 8543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:19.541822 8543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.547266 8543 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:46301
I20260927 07:09:19.547338 8671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:46301 every 8 connection(s)
I20260927 07:09:19.547622 8543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:19.554050 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 8543
I20260927 07:09:19.554157 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:19.554327 4793 external_mini_cluster.cc:1428] Setting key 142ffa1c3f39d95cb2d7d40f9c589bfb
I20260927 07:09:19.556834 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.548299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:38275 (local address 127.4.174.126:40071)
0927 07:09:19.548359 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:19.548361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:19.548584 (+ 223us) server_negotiation.cc:407] Connection header received
0927 07:09:19.549467 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:19.549469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:19.549480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:19.549554 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:19.550177 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.550720 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.551682 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.551871 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.551926 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.554010 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:19.554012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:19.554012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:19.554020 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:19.555563 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.555930 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.555932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.555932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.555990 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.556209 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.556210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.556210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.556337 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:19.556405 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:19.556704 (+ 299us) server_negotiation.cc:299] Negotiation successful
0927 07:09:19.556771 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 07:09:19.557125 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.548268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40071 (local address 127.4.174.66:38275)
0927 07:09:19.548425 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:19.548474 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:19.549354 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:19.549628 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:19.549631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:19.549884 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:19.550084 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:19.550088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.550788 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.550789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:19.551603 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:19.551605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.551962 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.551963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:19.552079 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.552553 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:19.552562 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:19.553920 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:19.555653 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:19.555658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:19.555660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:19.555868 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:19.556060 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:19.556061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:19.556061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:19.556151 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:19.556401 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:19.556406 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:19.556574 (+ 168us) client_negotiation.cc:769] Sending connection context
0927 07:09:19.556685 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 07:09:19.556846 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260927 07:09:19.557639 8672 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40071
I20260927 07:09:19.557722 8672 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:19.557888 8672 heartbeater.cc:511] Master 127.4.174.126:40071 requested a full tablet report, sending...
I20260927 07:09:19.558233 8347 ts_manager.cc:194] Registered new tserver with Master: ce111a5177c645eb877e2f4748e07bed (127.4.174.66:46301)
I20260927 07:09:19.558696 8347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:38275
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:19.578224 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40071
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:19.657040 8677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:19.657248 8677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:19.657279 8677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:19.658439 8677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:19.658484 8677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:19.658520 8677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:19.660041 8677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:19.660291 8677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:19.660498 8677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:19.665218 8685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:19.665304 8682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:19.665366 8677 server_base.cc:1034] running on GCE node
W20260927 07:09:19.665277 8683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:19.665652 8677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:19.665900 8677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:19.667040 8677 hybrid_clock.cc:648] HybridClock initialized: now 1790492959667015 us; error 37 us; skew 500 ppm
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:19.669370 8677 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:19.669799 8677 webserver.cc:460] Webserver started at http://127.4.174.67:42393/ using document root <none> and password file <none>
I20260927 07:09:19.670015 8677 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:19.670063 8677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:19.670193 8677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:19.671279 8677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "5bca52df95754d08baae9aff8f4e7253"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "f1255b8b22e369eb7b7ec0d536794287"
server_key_iv: "69e1f35efc3a37007437342be38e790c"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.671608 8677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "5bca52df95754d08baae9aff8f4e7253"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "f1255b8b22e369eb7b7ec0d536794287"
server_key_iv: "69e1f35efc3a37007437342be38e790c"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.672775 8677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 07:09:19.673412 8692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.673557 8677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:19.673636 8677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "5bca52df95754d08baae9aff8f4e7253"
format_stamp: "Formatted at 2026-09-27 07:09:19 on dist-test-slave-zw7p"
server_key: "f1255b8b22e369eb7b7ec0d536794287"
server_key_iv: "69e1f35efc3a37007437342be38e790c"
server_key_version: "encryptionkey@0"
I20260927 07:09:19.673693 8677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:19.725194 8677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:19.727038 8677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:19.727167 8677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:19.727377 8677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:19.727685 8677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:19.727725 8677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.727754 8677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:19.727778 8677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:19.733208 8677 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:35061
I20260927 07:09:19.733258 8805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:35061 every 8 connection(s)
I20260927 07:09:19.733546 8677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:19.742072 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.734188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:42613 (local address 127.4.174.126:40071)
0927 07:09:19.734244 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:19.734247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:19.734428 (+ 181us) server_negotiation.cc:407] Connection header received
0927 07:09:19.735361 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:19.735363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:19.735376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:19.735422 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:19.736040 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.736533 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.737489 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.737675 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.737728 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.739634 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:19.739635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:19.739636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:19.739643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:19.740881 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.741241 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.741242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.741242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.741294 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.741502 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.741503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.741504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.741608 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:19.741660 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:19.741909 (+ 249us) server_negotiation.cc:299] Negotiation successful
0927 07:09:19.741983 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 07:09:19.742235 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.734165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40071 (local address 127.4.174.67:42613)
0927 07:09:19.734314 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:19.734350 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:19.735252 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:19.735508 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:19.735511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:19.735754 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:19.735941 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:19.735945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.736612 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.736613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:19.737399 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:19.737400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.737738 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.737739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:19.737853 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.738293 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:19.738301 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:19.739547 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:19.740965 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:19.740971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:19.740973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:19.741175 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:19.741346 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:19.741347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:19.741347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:19.741448 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:19.741660 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:19.741666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:19.741795 (+ 129us) client_negotiation.cc:769] Sending connection context
0927 07:09:19.741878 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 07:09:19.742035 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260927 07:09:19.742651 8806 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40071
I20260927 07:09:19.742736 8806 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:19.742873 8806 heartbeater.cc:511] Master 127.4.174.126:40071 requested a full tablet report, sending...
I20260927 07:09:19.743192 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 8677
I20260927 07:09:19.743250 8347 ts_manager.cc:194] Registered new tserver with Master: 5bca52df95754d08baae9aff8f4e7253 (127.4.174.67:35061)
I20260927 07:09:19.743275 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:19.743448 4793 external_mini_cluster.cc:1428] Setting key db0f71a108c943c15154eaff1c5368ad
I20260927 07:09:19.743710 8347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:42613
I20260927 07:09:19.744201 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:19.765265 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.758287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.4.174.65:46867)
0927 07:09:19.758459 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:19.758464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:19.758475 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:09:19.758501 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:19.758503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:19.758516 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:19.758649 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:19.759134 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.759854 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.760595 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.760937 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.760999 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.763065 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:19.763075 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:19.763075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:19.763083 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:19.764333 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.764621 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.764623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.764623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.764671 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.764789 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.764789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.764789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.764905 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:19.765014 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:19.765130 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 07:09:19.765185 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:19.772814 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:19.766433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.4.174.126:40071)
0927 07:09:19.766484 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:19.766487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:19.766495 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:19.766535 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:19.766535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:19.766547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:19.766616 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:19.767056 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.767567 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.768230 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:19.768421 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:19.768474 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:19.770522 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:19.770523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:19.770524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:19.770531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:19.771809 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.772095 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.772096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.772096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.772148 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:19.772307 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:19.772308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:19.772308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:19.772453 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:19.772538 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:19.772637 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 07:09:19.772713 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260927 07:09:19.773025 8347 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50128
I20260927 07:09:19.773329 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 8409
I20260927 07:09:19.777227 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 8543
I20260927 07:09:19.781031 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 8677
I20260927 07:09:19.785046 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 8315
2026-09-27T07:09:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1757 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8818](info): set up 2 sockets
Sep 27 07:09:19 dist-test-slave-zw7p krb5kdc[8818](info): commencing operation
krb5kdc: starting...
W20260927 07:09:20.730269 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:20 dist-test-slave-zw7p krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:20.843452 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35841
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:35841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:20.925274 8831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:20.925483 8831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:20.925506 8831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:20.926703 8831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:20.926743 8831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:20.926756 8831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:20.926769 8831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:20.926779 8831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:20.928308 8831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:35841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:20.928511 8831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:20.928684 8831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:20.933633 8836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:20.933632 8839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:20.933643 8837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:20.933719 8831 server_base.cc:1034] running on GCE node
I20260927 07:09:20.934017 8831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:20.934301 8831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:20.935431 8831 hybrid_clock.cc:648] HybridClock initialized: now 1790492960935421 us; error 29 us; skew 500 ppm
Sep 27 07:09:20 dist-test-slave-zw7p krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492960, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:20.937799 8831 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:20.938138 8831 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:20.938191 8831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:20.938311 8831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:20.939410 8831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "5c0a308a4f7c4f48b3906de60c64f3ce"
format_stamp: "Formatted at 2026-09-27 07:09:20 on dist-test-slave-zw7p"
server_key: "1277e9844a754adfcc7a3923c39e081a"
server_key_iv: "4aa8487a4fb306c132281ac1e32cc251"
server_key_version: "encryptionkey@0"
I20260927 07:09:20.939754 8831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "5c0a308a4f7c4f48b3906de60c64f3ce"
format_stamp: "Formatted at 2026-09-27 07:09:20 on dist-test-slave-zw7p"
server_key: "1277e9844a754adfcc7a3923c39e081a"
server_key_iv: "4aa8487a4fb306c132281ac1e32cc251"
server_key_version: "encryptionkey@0"
I20260927 07:09:20.940969 8831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 07:09:20.941617 8845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:20.941767 8831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:20.941847 8831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "5c0a308a4f7c4f48b3906de60c64f3ce"
format_stamp: "Formatted at 2026-09-27 07:09:20 on dist-test-slave-zw7p"
server_key: "1277e9844a754adfcc7a3923c39e081a"
server_key_iv: "4aa8487a4fb306c132281ac1e32cc251"
server_key_version: "encryptionkey@0"
I20260927 07:09:20.941906 8831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:20.954903 8831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:20.956945 8831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:20.957078 8831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:20.960706 8831 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:35841
I20260927 07:09:20.960733 8897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:35841 every 8 connection(s)
I20260927 07:09:20.960994 8831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:20.961831 8898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:20.964092 8898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce: Bootstrap starting.
I20260927 07:09:20.964655 8898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:20.964922 8898 log.cc:824] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:20.965512 8898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce: No bootstrap required, opened a new log
I20260927 07:09:20.967069 8898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0a308a4f7c4f48b3906de60c64f3ce" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35841 } }
I20260927 07:09:20.967206 8898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:20.967228 8898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0a308a4f7c4f48b3906de60c64f3ce, State: Initialized, Role: FOLLOWER
I20260927 07:09:20.967368 8898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0a308a4f7c4f48b3906de60c64f3ce" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35841 } }
I20260927 07:09:20.967428 8898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:20.967449 8898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:20.967494 8898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:20.968184 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 8831
I20260927 07:09:20.968289 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:20.968308 8898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0a308a4f7c4f48b3906de60c64f3ce" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35841 } }
I20260927 07:09:20.968410 8898 leader_election.cc:302] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c0a308a4f7c4f48b3906de60c64f3ce; no voters:
I20260927 07:09:20.968490 4793 external_mini_cluster.cc:1428] Setting key 385dc3ae605f60f5e6501309e9b42230
I20260927 07:09:20.968564 8898 leader_election.cc:288] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:20.968705 8901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:20.968811 8898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:20.968889 8901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [term 1 LEADER]: Becoming Leader. State: Replica: 5c0a308a4f7c4f48b3906de60c64f3ce, State: Running, Role: LEADER
I20260927 07:09:20.969146 8901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0a308a4f7c4f48b3906de60c64f3ce" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35841 } }
I20260927 07:09:20.969509 8902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c0a308a4f7c4f48b3906de60c64f3ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0a308a4f7c4f48b3906de60c64f3ce" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35841 } } }
I20260927 07:09:20.969607 8902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:20.969843 8912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:20.970363 8912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:20.970343 8903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c0a308a4f7c4f48b3906de60c64f3ce. Latest consensus state: current_term: 1 leader_uuid: "5c0a308a4f7c4f48b3906de60c64f3ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0a308a4f7c4f48b3906de60c64f3ce" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35841 } } }
I20260927 07:09:20.970408 8903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:20.972316 8912 catalog_manager.cc:1347] Generated new cluster ID: f7513ed65937429fb60ec408eaaf61ce
I20260927 07:09:20.972368 8912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:09:20 dist-test-slave-zw7p krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:20.980161 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:20.968949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.4.174.126:35841)
0927 07:09:20.969099 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:20.969109 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:20.969147 (+ 38us) server_negotiation.cc:407] Connection header received
0927 07:09:20.969276 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:20.969282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:20.969324 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:20.969429 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:20.971056 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:20.972191 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:20.973216 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:20.973885 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:20.974022 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:20.976675 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:20.976679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:20.976680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:20.976692 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:20.978512 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:20.978941 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:20.978944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:20.978944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:20.979051 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:20.979236 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:20.979237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:20.979237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:20.979523 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:20.979665 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:20.979769 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 07:09:20.979897 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260927 07:09:20.988243 8912 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:20.988974 8912 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:20.995694 8912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce: Generated new TSK 0
I20260927 07:09:20.995898 8912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:21.023303 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35841
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:21.102828 8924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:21.103044 8924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:21.103077 8924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:21.104251 8924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:21.104295 8924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:21.104331 8924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:21.105855 8924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:21.106103 8924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:21.106316 8924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:21.111083 8930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.111086 8929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.111125 8932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.111199 8924 server_base.cc:1034] running on GCE node
I20260927 07:09:21.111531 8924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:21.111749 8924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:21.112896 8924 hybrid_clock.cc:648] HybridClock initialized: now 1790492961112887 us; error 36 us; skew 500 ppm
Sep 27 07:09:21 dist-test-slave-zw7p krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492961, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:21.115235 8924 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:21.115576 8924 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:21.115628 8924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:21.115751 8924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:21.116775 8924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "9d5e48af98594a69a495e64794438e66"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "b45f78c2931b822e8b856ccca26a9d88"
server_key_iv: "b42348b03366d111d5449ae83d6aba6e"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.117095 8924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "9d5e48af98594a69a495e64794438e66"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "b45f78c2931b822e8b856ccca26a9d88"
server_key_iv: "b42348b03366d111d5449ae83d6aba6e"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.118242 8924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:21.118922 8938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.119077 8924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.119155 8924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "9d5e48af98594a69a495e64794438e66"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "b45f78c2931b822e8b856ccca26a9d88"
server_key_iv: "b42348b03366d111d5449ae83d6aba6e"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.119213 8924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:21.133525 8924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:21.135313 8924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:21.135449 8924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:21.135663 8924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:21.135959 8924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:21.135993 8924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.136021 8924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:21.136044 8924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.141369 8924 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:34191
I20260927 07:09:21.141414 9051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:34191 every 8 connection(s)
I20260927 07:09:21.141657 8924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:21 dist-test-slave-zw7p krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492961, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:21.148121 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 8924
I20260927 07:09:21.148214 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:21.148415 4793 external_mini_cluster.cc:1428] Setting key 9e7552e8b931a804a1af46e68840b7a2
I20260927 07:09:21.150549 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.142257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:52575 (local address 127.4.174.126:35841)
0927 07:09:21.142298 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.142300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.142523 (+ 223us) server_negotiation.cc:407] Connection header received
0927 07:09:21.143046 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.143047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.143063 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.143113 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:21.144024 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.144571 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.145605 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.145811 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.145866 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.147840 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:21.147841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:21.147842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:21.147851 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:21.149273 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:21.149622 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:21.149623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:21.149623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:21.149677 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:21.149883 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:21.149884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:21.149884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:21.150018 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:21.150084 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.150368 (+ 284us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.150460 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 07:09:21.150667 9054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.142246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35841 (local address 127.4.174.65:52575)
0927 07:09:21.142403 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.142440 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.142945 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.143182 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.143186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.143440 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:21.143927 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.143933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.144653 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.144654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.145525 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.145528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.145879 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.145879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.146020 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.146434 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:21.146441 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:21.147753 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:21.149342 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:21.149349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:21.149351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:21.149557 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:21.149729 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:21.149730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:21.149731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:21.149828 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:21.150104 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:21.150112 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:21.150242 (+ 130us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.150335 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.150457 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260927 07:09:21.151151 9052 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35841
I20260927 07:09:21.151240 9052 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:21.151381 9052 heartbeater.cc:511] Master 127.4.174.126:35841 requested a full tablet report, sending...
I20260927 07:09:21.151867 8862 ts_manager.cc:194] Registered new tserver with Master: 9d5e48af98594a69a495e64794438e66 (127.4.174.65:34191)
I20260927 07:09:21.152522 8862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:52575
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:21.173569 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35841
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:21.251690 9057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:21.251894 9057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:21.251924 9057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:21.253057 9057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:21.253098 9057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:21.253134 9057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:21.254613 9057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:21.254837 9057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:21.255039 9057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:21.259692 9062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.259770 9065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.259701 9063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.259963 9057 server_base.cc:1034] running on GCE node
I20260927 07:09:21.260169 9057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:21.260437 9057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:21.261600 9057 hybrid_clock.cc:648] HybridClock initialized: now 1790492961261587 us; error 29 us; skew 500 ppm
Sep 27 07:09:21 dist-test-slave-zw7p krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492961, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:21.263922 9057 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:21.264247 9057 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:21.264297 9057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:21.264433 9057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:21.265453 9057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "e3e99aa7de844e64992716face6a328c"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "c2255fffff6aadd24a197cbab54a89c0"
server_key_iv: "08713d2b80f4438c518880f0b0158526"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.265780 9057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "e3e99aa7de844e64992716face6a328c"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "c2255fffff6aadd24a197cbab54a89c0"
server_key_iv: "08713d2b80f4438c518880f0b0158526"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.267019 9057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 07:09:21.267678 9071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.267807 9057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:21.267886 9057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "e3e99aa7de844e64992716face6a328c"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "c2255fffff6aadd24a197cbab54a89c0"
server_key_iv: "08713d2b80f4438c518880f0b0158526"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.267951 9057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:21.287202 9057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:21.289044 9057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:21.289175 9057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:21.289386 9057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:21.289680 9057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:21.289712 9057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.289741 9057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:21.289763 9057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.294979 9057 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:42513
I20260927 07:09:21.295024 9184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:42513 every 8 connection(s)
I20260927 07:09:21.295264 9057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:21.298060 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 9057
I20260927 07:09:21.298161 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:21.298327 4793 external_mini_cluster.cc:1428] Setting key e80f75d5d54087f8603356909f60a3ea
Sep 27 07:09:21 dist-test-slave-zw7p krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492961, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:21.304265 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.295889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:45741 (local address 127.4.174.126:35841)
0927 07:09:21.295966 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.295968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.296126 (+ 158us) server_negotiation.cc:407] Connection header received
0927 07:09:21.296588 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.296589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.296599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.296649 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:21.297500 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.298045 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.299204 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.299482 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.299552 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.301565 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:21.301566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:21.301567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:21.301573 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:21.302909 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:21.303374 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:21.303375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:21.303376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:21.303435 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:21.303688 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:21.303688 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:21.303688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:21.303782 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:21.303824 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.304133 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.304197 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 07:09:21.304517 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.295861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35841 (local address 127.4.174.66:45741)
0927 07:09:21.296013 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.296048 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.296515 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.296702 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.296705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.296953 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:21.297435 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.297440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.298118 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.298119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.299133 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.299136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.299552 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.299552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.299646 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.300105 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:21.300113 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:21.301467 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:21.302985 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:21.302993 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:21.302995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:21.303305 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:21.303488 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:21.303489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:21.303490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:21.303632 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:21.303840 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:21.303845 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:21.304009 (+ 164us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.304124 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.304287 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260927 07:09:21.305045 9185 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35841
I20260927 07:09:21.305138 9185 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:21.305275 9185 heartbeater.cc:511] Master 127.4.174.126:35841 requested a full tablet report, sending...
I20260927 07:09:21.305657 8862 ts_manager.cc:194] Registered new tserver with Master: e3e99aa7de844e64992716face6a328c (127.4.174.66:42513)
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
I20260927 07:09:21.306165 8862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:45741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:21.321310 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35841
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:21.400631 9190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:21.400859 9190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:21.400887 9190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:21.402034 9190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:21.402078 9190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:21.402105 9190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:21.403626 9190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:21.403864 9190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:21.404070 9190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:21.408918 9195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.409082 9190 server_base.cc:1034] running on GCE node
W20260927 07:09:21.408993 9198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.408922 9196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.409409 9190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:21.409684 9190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:21.410835 9190 hybrid_clock.cc:648] HybridClock initialized: now 1790492961410819 us; error 24 us; skew 500 ppm
Sep 27 07:09:21 dist-test-slave-zw7p krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492961, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:21.413285 9190 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:21.413647 9190 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:21.413702 9190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:21.413836 9190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:21.414934 9190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "6d0a01b8522a4a77a2c7114ebe77b51d"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "bae34aa413765e5aabcda363706af86f"
server_key_iv: "327802a456c75eb515e942063ba2a7e4"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.415251 9190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "6d0a01b8522a4a77a2c7114ebe77b51d"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "bae34aa413765e5aabcda363706af86f"
server_key_iv: "327802a456c75eb515e942063ba2a7e4"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.416428 9190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:21.417098 9204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.417249 9190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:21.417330 9190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "6d0a01b8522a4a77a2c7114ebe77b51d"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "bae34aa413765e5aabcda363706af86f"
server_key_iv: "327802a456c75eb515e942063ba2a7e4"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.417390 9190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:21.434295 9190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:21.436463 9190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:21.436599 9190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:21.436811 9190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:21.437129 9190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:21.437161 9190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.437189 9190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:21.437211 9190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.442660 9190 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:40463
I20260927 07:09:21.442699 9317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:40463 every 8 connection(s)
I20260927 07:09:21.442958 9190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:21.445525 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 9190
I20260927 07:09:21.445647 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:21.445845 4793 external_mini_cluster.cc:1428] Setting key 90c9608e395c747081e789495a40d245
Sep 27 07:09:21 dist-test-slave-zw7p krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492961, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:21.451571 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.443605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:54401 (local address 127.4.174.126:35841)
0927 07:09:21.443650 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.443652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.443841 (+ 189us) server_negotiation.cc:407] Connection header received
0927 07:09:21.444332 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.444333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.444343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.444409 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:21.445343 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.446084 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.447292 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.447484 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.447536 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.449445 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:21.449447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:21.449447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:21.449454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:21.450511 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:21.450865 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:21.450866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:21.450867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:21.450920 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:21.451093 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:21.451094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:21.451094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:21.451189 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:21.451233 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.451445 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.451507 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:09:21.451747 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.443578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35841 (local address 127.4.174.67:54401)
0927 07:09:21.443720 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.443762 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.444255 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.444462 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.444465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.444742 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:21.445258 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.445265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.446179 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.446180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.447221 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.447223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.447543 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.447544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.447642 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.448102 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:21.448109 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:21.449366 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:21.450594 (+ 1228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:21.450599 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:21.450601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:21.450800 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:21.450964 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:21.450964 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:21.450965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:21.451054 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:21.451229 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:21.451231 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:21.451344 (+ 113us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.451419 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.451537 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260927 07:09:21.452152 9318 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35841
I20260927 07:09:21.452235 9318 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:21.452379 9318 heartbeater.cc:511] Master 127.4.174.126:35841 requested a full tablet report, sending...
I20260927 07:09:21.452714 8862 ts_manager.cc:194] Registered new tserver with Master: 6d0a01b8522a4a77a2c7114ebe77b51d (127.4.174.67:40463)
I20260927 07:09:21.453047 8862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:54401
I20260927 07:09:21.457379 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:09:21.464071 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.458627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34282 (local address 127.4.174.126:35841)
0927 07:09:21.458673 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.458676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.458768 (+ 92us) server_negotiation.cc:407] Connection header received
0927 07:09:21.458829 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.458830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.458840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.458884 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:21.459420 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.459888 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.460596 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.460796 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.460850 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.461946 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:21.461947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:21.461948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:21.461955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:21.463030 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:21.463327 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:21.463328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:21.463329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:21.463382 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:21.463552 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:21.463553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:21.463553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:21.463698 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:21.463784 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.463903 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.463980 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 07:09:21.465489 8862 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:21.467171 8862 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:21.473847 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.469462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:34191 (local address 127.0.0.1:58462)
0927 07:09:21.470110 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.470123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.470173 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.470409 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.470410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.470411 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:21.470439 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.470440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.471728 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.471730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.472587 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.472590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.473504 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.473505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.473592 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.473610 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.473691 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.473749 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":1,"thread_start_us":36,"threads_started":1}
I20260927 07:09:21.473856 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.469573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.4.174.65:34191)
0927 07:09:21.469786 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.469790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.470262 (+ 472us) server_negotiation.cc:407] Connection header received
0927 07:09:21.470311 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.470313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.470327 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.470409 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:21.470540 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.471623 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.472670 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.473402 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.473480 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.473550 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.473728 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.473771 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":84,"threads_started":1}
I20260927 07:09:21.473923 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.469372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45340 (local address 127.4.174.67:40463)
0927 07:09:21.469549 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.469554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.469665 (+ 111us) server_negotiation.cc:407] Connection header received
0927 07:09:21.469796 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.469799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.469816 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.469890 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:21.470161 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.471339 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.472839 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.473518 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.473581 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.473628 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.473800 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.473844 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260927 07:09:21.474082 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.469256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:40463 (local address 127.0.0.1:45340)
0927 07:09:21.469596 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.469611 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.469693 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.469986 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.469989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.469991 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:21.470092 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.470095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.471439 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.471440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.472756 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.472759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.473599 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.473600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.473652 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.473654 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.473704 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.473756 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":66,"thread_start_us":58,"threads_started":1}
I20260927 07:09:21.474850 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.469243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:42513 (local address 127.0.0.1:39414)
0927 07:09:21.469563 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.469608 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.469673 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.470225 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.470226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.470228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:21.470267 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.470268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.471378 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.471379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.472756 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.472759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.474613 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.474614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.474678 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.474683 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.474736 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.474780 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":86,"thread_start_us":80,"threads_started":1}
I20260927 07:09:21.474900 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.469606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.4.174.66:42513)
0927 07:09:21.469876 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.469881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.469890 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:21.469924 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.469926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.469938 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.470002 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:21.470337 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.471288 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.473661 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.474438 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.474504 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.474580 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.474770 (+ 190us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.474818 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":77,"threads_started":1}
I20260927 07:09:21.475072 8986 tablet_service.cc:1467] Processing CreateTablet for tablet 9fdc560dab884120b3cb7d123beeb9ad (DEFAULT_TABLE table=test-table [id=94f2adb83ed5472eb75e4bacd3c3df22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:21.475081 9252 tablet_service.cc:1467] Processing CreateTablet for tablet 9fdc560dab884120b3cb7d123beeb9ad (DEFAULT_TABLE table=test-table [id=94f2adb83ed5472eb75e4bacd3c3df22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:21.475428 9252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fdc560dab884120b3cb7d123beeb9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:21.475423 8986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fdc560dab884120b3cb7d123beeb9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:21.475785 9119 tablet_service.cc:1467] Processing CreateTablet for tablet 9fdc560dab884120b3cb7d123beeb9ad (DEFAULT_TABLE table=test-table [id=94f2adb83ed5472eb75e4bacd3c3df22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:21.476058 9119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fdc560dab884120b3cb7d123beeb9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:21.477983 9332 tablet_bootstrap.cc:492] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: Bootstrap starting.
I20260927 07:09:21.478497 9332 tablet_bootstrap.cc:654] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:21.478518 9333 tablet_bootstrap.cc:492] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66: Bootstrap starting.
I20260927 07:09:21.479089 9332 log.cc:824] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:21.479162 9333 tablet_bootstrap.cc:654] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:21.479239 9334 tablet_bootstrap.cc:492] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Bootstrap starting.
I20260927 07:09:21.479676 9333 log.cc:824] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:21.479853 9334 tablet_bootstrap.cc:654] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:21.479913 9332 tablet_bootstrap.cc:492] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: No bootstrap required, opened a new log
I20260927 07:09:21.479996 9332 ts_tablet_manager.cc:1397] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 07:09:21.480410 9333 tablet_bootstrap.cc:492] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66: No bootstrap required, opened a new log
I20260927 07:09:21.480440 9334 log.cc:824] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:21.480536 9333 ts_tablet_manager.cc:1397] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:21.481145 9334 tablet_bootstrap.cc:492] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: No bootstrap required, opened a new log
I20260927 07:09:21.481213 9334 ts_tablet_manager.cc:1397] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:21.481700 9332 raft_consensus.cc:348] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.481830 9332 raft_consensus.cc:374] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:21.481859 9332 raft_consensus.cc:729] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0a01b8522a4a77a2c7114ebe77b51d, State: Initialized, Role: FOLLOWER
I20260927 07:09:21.482002 9332 consensus_queue.cc:260] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.482256 9332 ts_tablet_manager.cc:1428] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:09:21.482271 9318 heartbeater.cc:503] Master 127.4.174.126:35841 was elected leader, sending a full tablet report...
I20260927 07:09:21.482337 9333 raft_consensus.cc:348] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.482496 9333 raft_consensus.cc:374] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:21.482527 9333 raft_consensus.cc:729] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d5e48af98594a69a495e64794438e66, State: Initialized, Role: FOLLOWER
I20260927 07:09:21.482725 9333 consensus_queue.cc:260] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.483029 9333 ts_tablet_manager.cc:1428] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:09:21.483057 9052 heartbeater.cc:503] Master 127.4.174.126:35841 was elected leader, sending a full tablet report...
I20260927 07:09:21.483125 9334 raft_consensus.cc:348] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.483258 9334 raft_consensus.cc:374] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:21.483289 9334 raft_consensus.cc:729] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e99aa7de844e64992716face6a328c, State: Initialized, Role: FOLLOWER
I20260927 07:09:21.483429 9334 consensus_queue.cc:260] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.483690 9334 ts_tablet_manager.cc:1428] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:09:21.483697 9185 heartbeater.cc:503] Master 127.4.174.126:35841 was elected leader, sending a full tablet report...
W20260927 07:09:21.545764 9186 tablet.cc:2369] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:21.560982 9338 raft_consensus.cc:482] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:21.561093 9338 raft_consensus.cc:504] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.561429 9338 leader_election.cc:288] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3e99aa7de844e64992716face6a328c (127.4.174.66:42513), 9d5e48af98594a69a495e64794438e66 (127.4.174.65:34191)
I20260927 07:09:21.565173 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.561678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:45435 (local address 127.4.174.66:42513)
0927 07:09:21.561751 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.561753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.561762 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:21.561787 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.561788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.561797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.561840 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:21.562021 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.562799 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.564270 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.564832 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.564906 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.564916 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.565061 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.565099 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:09:21.565173 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.561809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:33701 (local address 127.4.174.65:34191)
0927 07:09:21.561851 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.561852 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.561946 (+ 94us) server_negotiation.cc:407] Connection header received
0927 07:09:21.562009 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.562010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.562018 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.562070 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:21.562169 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.562862 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.564264 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.564832 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.564912 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.564919 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.565061 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.565099 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:09:21.565184 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.561603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:42513 (local address 127.4.174.67:45435)
0927 07:09:21.561659 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.561673 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.561722 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.561884 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.561886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.561888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:21.561958 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.561960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.562886 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.562887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.564171 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.564175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.564911 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.564912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.564999 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.565001 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.565049 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.565093 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":29}
I20260927 07:09:21.565196 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.561713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:34191 (local address 127.4.174.67:33701)
0927 07:09:21.561879 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.561889 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.561947 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.562071 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.562072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.562073 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:21.562119 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.562120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.562945 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.562946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.564171 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:21.564175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.564912 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.564913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:21.564989 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.564994 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.565049 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.565093 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
I20260927 07:09:21.565570 9139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fdc560dab884120b3cb7d123beeb9ad" candidate_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e99aa7de844e64992716face6a328c" is_pre_election: true
I20260927 07:09:21.565706 9139 raft_consensus.cc:2454] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0a01b8522a4a77a2c7114ebe77b51d in term 0.
I20260927 07:09:21.565948 9206 leader_election.cc:302] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d0a01b8522a4a77a2c7114ebe77b51d, e3e99aa7de844e64992716face6a328c; no voters:
I20260927 07:09:21.566045 9006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fdc560dab884120b3cb7d123beeb9ad" candidate_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d5e48af98594a69a495e64794438e66" is_pre_election: true
I20260927 07:09:21.566093 9338 raft_consensus.cc:2781] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:21.566159 9338 raft_consensus.cc:482] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:21.566191 9338 raft_consensus.cc:3037] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:21.566208 9006 raft_consensus.cc:2454] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0a01b8522a4a77a2c7114ebe77b51d in term 0.
I20260927 07:09:21.566952 9338 raft_consensus.cc:504] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.567071 9338 leader_election.cc:288] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [CANDIDATE]: Term 1 election: Requested vote from peers e3e99aa7de844e64992716face6a328c (127.4.174.66:42513), 9d5e48af98594a69a495e64794438e66 (127.4.174.65:34191)
I20260927 07:09:21.567216 9139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fdc560dab884120b3cb7d123beeb9ad" candidate_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e99aa7de844e64992716face6a328c"
I20260927 07:09:21.567269 9006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fdc560dab884120b3cb7d123beeb9ad" candidate_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d5e48af98594a69a495e64794438e66"
I20260927 07:09:21.567291 9139 raft_consensus.cc:3037] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:21.567348 9006 raft_consensus.cc:3037] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:21.568002 9139 raft_consensus.cc:2454] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0a01b8522a4a77a2c7114ebe77b51d in term 1.
I20260927 07:09:21.568011 9006 raft_consensus.cc:2454] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0a01b8522a4a77a2c7114ebe77b51d in term 1.
I20260927 07:09:21.568157 9206 leader_election.cc:302] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d0a01b8522a4a77a2c7114ebe77b51d, e3e99aa7de844e64992716face6a328c; no voters:
I20260927 07:09:21.568270 9338 raft_consensus.cc:2781] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:21.568428 9338 raft_consensus.cc:686] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 1 LEADER]: Becoming Leader. State: Replica: 6d0a01b8522a4a77a2c7114ebe77b51d, State: Running, Role: LEADER
I20260927 07:09:21.568517 9338 consensus_queue.cc:237] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } }
I20260927 07:09:21.569372 8862 catalog_manager.cc:5495] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d0a01b8522a4a77a2c7114ebe77b51d (127.4.174.67). New cstate: current_term: 1 leader_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0a01b8522a4a77a2c7114ebe77b51d" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:21.587854 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.585570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.4.174.67:40463)
0927 07:09:21.585626 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.585628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.585636 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:21.585671 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.585672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.585690 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.585750 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:21.585885 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.586361 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.587180 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:21.587379 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:21.587426 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:21.587546 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:21.587680 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:21.587738 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.587756 (+ 18us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.587772 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:21.589632 9139 raft_consensus.cc:1261] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 1 FOLLOWER]: Refusing update from remote peer 6d0a01b8522a4a77a2c7114ebe77b51d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:21.589632 9006 raft_consensus.cc:1261] T 9fdc560dab884120b3cb7d123beeb9ad P 9d5e48af98594a69a495e64794438e66 [term 1 FOLLOWER]: Refusing update from remote peer 6d0a01b8522a4a77a2c7114ebe77b51d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:21.589991 9342 consensus_queue.cc:1035] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e99aa7de844e64992716face6a328c" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 42513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:21.590127 9338 consensus_queue.cc:1035] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d5e48af98594a69a495e64794438e66" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:21.592383 9345 mvcc.cc:205] Tried to move back new op lower bound from 7333859170668453888 to 7333859170585329664. Current Snapshot: MvccSnapshot[applied={T|T < 7333859170668453888}]
I20260927 07:09:21.593255 9346 mvcc.cc:205] Tried to move back new op lower bound from 7333859170668453888 to 7333859170585329664. Current Snapshot: MvccSnapshot[applied={T|T < 7333859170668453888}]
I20260927 07:09:21.593906 9348 mvcc.cc:205] Tried to move back new op lower bound from 7333859170668453888 to 7333859170585329664. Current Snapshot: MvccSnapshot[applied={T|T < 7333859170668453888}]
I20260927 07:09:21.595299 8861 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:21.595402 8861 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:21.596082 8861 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c0a308a4f7c4f48b3906de60c64f3ce: Sending DeleteTablet for 3 replicas of tablet 9fdc560dab884120b3cb7d123beeb9ad
I20260927 07:09:21.596401 9252 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fdc560dab884120b3cb7d123beeb9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:21 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:45340
I20260927 07:09:21.596473 9119 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fdc560dab884120b3cb7d123beeb9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:21 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:39414
I20260927 07:09:21.596638 9354 tablet_replica.cc:330] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: stopping tablet replica
I20260927 07:09:21.596673 8986 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fdc560dab884120b3cb7d123beeb9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:21 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:58462
I20260927 07:09:21.596755 9354 raft_consensus.cc:2229] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:21.596863 9354 raft_consensus.cc:2258] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:21.597054 9355 tablet_replica.cc:330] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: stopping tablet replica
I20260927 07:09:21.597052 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 8924
I20260927 07:09:21.597231 9355 raft_consensus.cc:2229] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:21.597365 9355 raft_consensus.cc:2258] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:21.597368 9354 ts_tablet_manager.cc:1905] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:21.598115 9355 ts_tablet_manager.cc:1905] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:21.599378 9354 ts_tablet_manager.cc:1918] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:21.599522 9354 log.cc:1196] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/9fdc560dab884120b3cb7d123beeb9ad
I20260927 07:09:21.599565 9355 ts_tablet_manager.cc:1918] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:21.599730 9355 log.cc:1196] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/9fdc560dab884120b3cb7d123beeb9ad
I20260927 07:09:21.599839 9354 ts_tablet_manager.cc:1939] T 9fdc560dab884120b3cb7d123beeb9ad P 6d0a01b8522a4a77a2c7114ebe77b51d: Deleting consensus metadata
I20260927 07:09:21.600106 9355 ts_tablet_manager.cc:1939] T 9fdc560dab884120b3cb7d123beeb9ad P e3e99aa7de844e64992716face6a328c: Deleting consensus metadata
I20260927 07:09:21.600265 8849 catalog_manager.cc:4841] TS 6d0a01b8522a4a77a2c7114ebe77b51d (127.4.174.67:40463): tablet 9fdc560dab884120b3cb7d123beeb9ad (table test-table [id=94f2adb83ed5472eb75e4bacd3c3df22]) successfully deleted
I20260927 07:09:21.600507 8847 catalog_manager.cc:4841] TS e3e99aa7de844e64992716face6a328c (127.4.174.66:42513): tablet 9fdc560dab884120b3cb7d123beeb9ad (table test-table [id=94f2adb83ed5472eb75e4bacd3c3df22]) successfully deleted
W20260927 07:09:21.603188 8847 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:34191 (error 108)
I20260927 07:09:21.603408 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 9057
I20260927 07:09:21.603999 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.603575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:34191 (local address 127.0.0.1:58470)
0927 07:09:21.603919 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.603966 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320}
W20260927 07:09:21.604300 8847 catalog_manager.cc:4584] TS 9d5e48af98594a69a495e64794438e66 (127.4.174.65:34191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9fdc560dab884120b3cb7d123beeb9ad: Network error: Client connection negotiation failed: client connection to 127.4.174.65:34191: connect: Connection refused (error 111)
I20260927 07:09:21.607416 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 9190
I20260927 07:09:21.611212 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 8831
2026-09-27T07:09:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1826 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-27T07:09:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:21Z Disabled control of system clock
I20260927 07:09:21.636224 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:45955
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:45955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 07:09:21.714648 9361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:21.714838 9361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:21.714857 9361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:21.715937 9361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:21.715969 9361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:21.715981 9361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:21.715991 9361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:21.716008 9361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:21.717250 9361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:45955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:45955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:21.717437 9361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:21.717656 9361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:21.722079 9369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.722256 9361 server_base.cc:1034] running on GCE node
W20260927 07:09:21.722084 9367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.722072 9366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.722633 9361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:21.722909 9361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:21.724040 9361 hybrid_clock.cc:648] HybridClock initialized: now 1790492961724029 us; error 28 us; skew 500 ppm
I20260927 07:09:21.725170 9361 webserver.cc:460] Webserver started at http://127.4.174.126:41029/ using document root <none> and password file <none>
I20260927 07:09:21.725381 9361 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:21.725430 9361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:21.725548 9361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:21.726640 9361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "eefb61d70eeb4e7796e80ef90287377e"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "0bc068668dec4e995d71c8f0642e1f1d"
server_key_iv: "cba58e22a542ec83de70dda5c06e1ce5"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.726979 9361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "eefb61d70eeb4e7796e80ef90287377e"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "0bc068668dec4e995d71c8f0642e1f1d"
server_key_iv: "cba58e22a542ec83de70dda5c06e1ce5"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.728156 9361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:21.728853 9375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.729005 9361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:21.729080 9361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "eefb61d70eeb4e7796e80ef90287377e"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "0bc068668dec4e995d71c8f0642e1f1d"
server_key_iv: "cba58e22a542ec83de70dda5c06e1ce5"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.729147 9361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:21.732369 9361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:21.732604 9361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:21.732707 9361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:21.736251 9361 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:45955
I20260927 07:09:21.736295 9427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:45955 every 8 connection(s)
I20260927 07:09:21.736585 9361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:21.737234 9428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:21.739877 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 9361
I20260927 07:09:21.739995 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:21.740186 4793 external_mini_cluster.cc:1428] Setting key 21ea424ca7c664b3775be2da4e043537
I20260927 07:09:21.740203 9428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e: Bootstrap starting.
I20260927 07:09:21.740868 9428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:21.741158 9428 log.cc:824] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:21.741906 9428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e: No bootstrap required, opened a new log
I20260927 07:09:21.742776 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.740715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.4.174.126:45955)
0927 07:09:21.740890 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.740899 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.740924 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:09:21.741367 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.741376 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.741455 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.741609 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:21.742323 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:21.742326 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:21.742326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:21.742330 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:21.742371 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:21.742381 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:21.742449 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.742518 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.742587 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260927 07:09:21.743609 9428 raft_consensus.cc:348] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb61d70eeb4e7796e80ef90287377e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45955 } }
I20260927 07:09:21.743753 9428 raft_consensus.cc:374] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:21.743788 9428 raft_consensus.cc:729] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eefb61d70eeb4e7796e80ef90287377e, State: Initialized, Role: FOLLOWER
I20260927 07:09:21.743908 9428 consensus_queue.cc:260] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb61d70eeb4e7796e80ef90287377e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45955 } }
I20260927 07:09:21.743984 9428 raft_consensus.cc:388] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:21.744019 9428 raft_consensus.cc:482] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:21.744069 9428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:21.744874 9428 raft_consensus.cc:504] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb61d70eeb4e7796e80ef90287377e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45955 } }
I20260927 07:09:21.744975 9428 leader_election.cc:302] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eefb61d70eeb4e7796e80ef90287377e; no voters:
I20260927 07:09:21.745131 9428 leader_election.cc:288] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:21.745172 9433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:21.745321 9433 raft_consensus.cc:686] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [term 1 LEADER]: Becoming Leader. State: Replica: eefb61d70eeb4e7796e80ef90287377e, State: Running, Role: LEADER
I20260927 07:09:21.745398 9428 sys_catalog.cc:564] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:21.745419 9433 consensus_queue.cc:237] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb61d70eeb4e7796e80ef90287377e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45955 } }
I20260927 07:09:21.745698 9433 sys_catalog.cc:455] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eefb61d70eeb4e7796e80ef90287377e. Latest consensus state: current_term: 1 leader_uuid: "eefb61d70eeb4e7796e80ef90287377e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb61d70eeb4e7796e80ef90287377e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45955 } } }
I20260927 07:09:21.745805 9433 sys_catalog.cc:458] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:21.745708 9435 sys_catalog.cc:455] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eefb61d70eeb4e7796e80ef90287377e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb61d70eeb4e7796e80ef90287377e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45955 } } }
I20260927 07:09:21.745962 9435 sys_catalog.cc:458] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:21.746318 9442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:21.746778 9442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:21.748140 9442 catalog_manager.cc:1347] Generated new cluster ID: d22c0ee4db234e5d9d3fd2a1bf5cc1db
I20260927 07:09:21.748190 9442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:21.761723 9442 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:21.761904 9442 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:21.766398 9442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e: Generated new TSK 0
I20260927 07:09:21.766605 9442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:09:21.768529 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45955
--builtin_ntp_servers=127.4.174.84:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 07:09:21.849814 9452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:21.850023 9452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:21.850046 9452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:21.851266 9452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:21.851307 9452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:21.851336 9452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:21.852639 9452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:21.852883 9452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:21.853180 9452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:21.858850 9460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.858860 9458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.858915 9457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.859095 9452 server_base.cc:1034] running on GCE node
I20260927 07:09:21.859392 9452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:21.859620 9452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:21.860787 9452 hybrid_clock.cc:648] HybridClock initialized: now 1790492961860765 us; error 39 us; skew 500 ppm
I20260927 07:09:21.862006 9452 webserver.cc:460] Webserver started at http://127.4.174.65:36271/ using document root <none> and password file <none>
I20260927 07:09:21.862236 9452 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:21.862286 9452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:21.862412 9452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:21.863497 9452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "507f4c54ca3f4b61b412ba1491c686bf"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "5a3396949e4902dfcf7380aad4a10d82"
server_key_iv: "ce7c55ede392686076d7d17df03b7a9e"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.863821 9452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "507f4c54ca3f4b61b412ba1491c686bf"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "5a3396949e4902dfcf7380aad4a10d82"
server_key_iv: "ce7c55ede392686076d7d17df03b7a9e"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.865077 9452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:21.865794 9466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.865952 9452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:21.866042 9452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "507f4c54ca3f4b61b412ba1491c686bf"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "5a3396949e4902dfcf7380aad4a10d82"
server_key_iv: "ce7c55ede392686076d7d17df03b7a9e"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.866106 9452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:21.869313 9452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:21.869557 9452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:21.869675 9452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:21.869858 9452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:21.870150 9452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:21.870183 9452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.870209 9452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:21.870234 9452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.875468 9452 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:35205
I20260927 07:09:21.875504 9579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:35205 every 8 connection(s)
I20260927 07:09:21.875815 9452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:21.878612 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.876457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:46003 (local address 127.4.174.126:45955)
0927 07:09:21.876509 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.876511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.876742 (+ 231us) server_negotiation.cc:407] Connection header received
0927 07:09:21.877176 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.877176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.877181 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.877258 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:21.878365 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:21.878367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:21.878367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:21.878371 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:21.878387 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:21.878391 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:21.878446 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.878509 (+ 63us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.878521 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 07:09:21.878785 9582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.876446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45955 (local address 127.4.174.65:46003)
0927 07:09:21.876614 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.876654 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.877098 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.877322 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.877325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.877333 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:21.878202 (+ 869us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:21.878202 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:21.878230 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:21.878259 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:21.878450 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:21.878453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:21.878456 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.878507 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.878581 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260927 07:09:21.879182 9580 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45955
I20260927 07:09:21.879264 9580 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:21.879377 9580 heartbeater.cc:511] Master 127.4.174.126:45955 requested a full tablet report, sending...
I20260927 07:09:21.879812 9392 ts_manager.cc:194] Registered new tserver with Master: 507f4c54ca3f4b61b412ba1491c686bf (127.4.174.65:35205)
I20260927 07:09:21.883313 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 9452
I20260927 07:09:21.883389 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:21.883530 4793 external_mini_cluster.cc:1428] Setting key 7019bcbeb46328f5e559aa80fe8b27a8
I20260927 07:09:21.884344 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45955
--builtin_ntp_servers=127.4.174.84:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 07:09:21.963325 9583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:21.963536 9583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:21.963559 9583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:21.964704 9583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:21.964747 9583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:21.964772 9583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:21.966094 9583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:21.966292 9583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:21.966583 9583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:21.971451 9591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:21.971446 9589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.971542 9583 server_base.cc:1034] running on GCE node
W20260927 07:09:21.971484 9588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:21.971881 9583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:21.972134 9583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:21.973281 9583 hybrid_clock.cc:648] HybridClock initialized: now 1790492961973268 us; error 29 us; skew 500 ppm
I20260927 07:09:21.974493 9583 webserver.cc:460] Webserver started at http://127.4.174.66:37959/ using document root <none> and password file <none>
I20260927 07:09:21.974757 9583 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:21.974809 9583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:21.974934 9583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:21.976003 9583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "487aa1fe3bac40c087b5506e941a68fd"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "59756bbe2287b0707e273c66978b78f1"
server_key_iv: "39f9df06894e983dfc308c88601acafb"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.976337 9583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "487aa1fe3bac40c087b5506e941a68fd"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "59756bbe2287b0707e273c66978b78f1"
server_key_iv: "39f9df06894e983dfc308c88601acafb"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.977581 9583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:21.978296 9597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.978492 9583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:21.978600 9583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "487aa1fe3bac40c087b5506e941a68fd"
format_stamp: "Formatted at 2026-09-27 07:09:21 on dist-test-slave-zw7p"
server_key: "59756bbe2287b0707e273c66978b78f1"
server_key_iv: "39f9df06894e983dfc308c88601acafb"
server_key_version: "encryptionkey@0"
I20260927 07:09:21.978670 9583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:21.981906 9583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:21.982143 9583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:21.982259 9583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:21.982453 9583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:21.982771 9583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:21.982805 9583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.982825 9583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:21.982849 9583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:21.988044 9583 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:43571
I20260927 07:09:21.988098 9710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:43571 every 8 connection(s)
I20260927 07:09:21.988387 9583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:21.988672 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 9583
I20260927 07:09:21.988770 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:21.988953 4793 external_mini_cluster.cc:1428] Setting key 735f419408ad9a5a540d164cbda152db
I20260927 07:09:21.989992 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45955
--builtin_ntp_servers=127.4.174.84:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 07:09:21.991715 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.989079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:39447 (local address 127.4.174.126:45955)
0927 07:09:21.989222 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:21.989225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:21.989469 (+ 244us) server_negotiation.cc:407] Connection header received
0927 07:09:21.990038 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:21.990040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:21.990048 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:21.990125 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:21.991496 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:21.991497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:21.991497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:21.991502 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:21.991518 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:21.991522 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:21.991570 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:21.991631 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 07:09:21.991645 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260927 07:09:21.991973 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:21.989066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45955 (local address 127.4.174.66:39447)
0927 07:09:21.989315 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:21.989358 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:21.989919 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:21.990257 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:21.990263 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:21.990272 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:21.991333 (+ 1061us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:21.991334 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:21.991379 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:21.991419 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:21.991568 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:21.991572 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:21.991576 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:21.991643 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 07:09:21.991706 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
I20260927 07:09:21.992391 9711 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45955
I20260927 07:09:21.992496 9711 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:21.992667 9711 heartbeater.cc:511] Master 127.4.174.126:45955 requested a full tablet report, sending...
I20260927 07:09:21.993010 9392 ts_manager.cc:194] Registered new tserver with Master: 487aa1fe3bac40c087b5506e941a68fd (127.4.174.66:43571)
W20260927 07:09:22.071864 9714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:22.072089 9714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:22.072113 9714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:22.073329 9714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:22.073374 9714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:22.073400 9714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:22.074788 9714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:22.075038 9714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:22.075337 9714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:22.080054 9719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:22.080062 9720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:22.080230 9722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:22.080457 9714 server_base.cc:1034] running on GCE node
I20260927 07:09:22.080724 9714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:22.080971 9714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:22.082154 9714 hybrid_clock.cc:648] HybridClock initialized: now 1790492962082140 us; error 26 us; skew 500 ppm
I20260927 07:09:22.083424 9714 webserver.cc:460] Webserver started at http://127.4.174.67:40887/ using document root <none> and password file <none>
I20260927 07:09:22.083662 9714 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:22.083727 9714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:22.083861 9714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:22.084916 9714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "da3376110c9948bc8edec75cab991789"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "e149085291ac1a4271a3c1aff3086f4a"
server_key_iv: "f2731116e60bcd4fc4955ef964092f1a"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.085250 9714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "da3376110c9948bc8edec75cab991789"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "e149085291ac1a4271a3c1aff3086f4a"
server_key_iv: "f2731116e60bcd4fc4955ef964092f1a"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.086455 9714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:22.087332 9728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:22.087543 9714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:22.087631 9714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "da3376110c9948bc8edec75cab991789"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "e149085291ac1a4271a3c1aff3086f4a"
server_key_iv: "f2731116e60bcd4fc4955ef964092f1a"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.087711 9714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:22.091114 9714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:22.091362 9714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:22.091477 9714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:22.091657 9714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:22.091960 9714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:22.091993 9714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:22.092021 9714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:22.092046 9714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:22.097307 9714 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:33781
I20260927 07:09:22.097357 9841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:33781 every 8 connection(s)
I20260927 07:09:22.097653 9714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:22.100375 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.098333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:42531 (local address 127.4.174.126:45955)
0927 07:09:22.098391 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.098393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.098629 (+ 236us) server_negotiation.cc:407] Connection header received
0927 07:09:22.099125 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.099126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.099130 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.099167 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.100190 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.100191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.100191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.100195 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.100208 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.100212 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.100255 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.100309 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.100321 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 07:09:22.100553 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.098328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45955 (local address 127.4.174.67:42531)
0927 07:09:22.098497 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.098535 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:22.099025 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:22.099231 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:22.099235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:22.099243 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:22.100061 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:22.100062 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:22.100089 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:22.100115 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:22.100254 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:22.100257 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:22.100259 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:09:22.100313 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 07:09:22.100363 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260927 07:09:22.100960 9842 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45955
I20260927 07:09:22.101051 9842 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:22.101192 9842 heartbeater.cc:511] Master 127.4.174.126:45955 requested a full tablet report, sending...
I20260927 07:09:22.101538 9392 ts_manager.cc:194] Registered new tserver with Master: da3376110c9948bc8edec75cab991789 (127.4.174.67:33781)
I20260927 07:09:22.104862 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 9714
I20260927 07:09:22.104943 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:22.105087 4793 external_mini_cluster.cc:1428] Setting key cb632278bb8630685b89eb85d9224560
I20260927 07:09:22.105824 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:09:22.108028 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.107047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40324 (local address 127.4.174.126:45955)
0927 07:09:22.107092 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.107094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.107186 (+ 92us) server_negotiation.cc:407] Connection header received
0927 07:09:22.107249 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.107249 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.107254 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.107295 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.107835 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.107836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.107836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.107840 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.107854 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.107858 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.107898 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.107950 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.107964 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:22.109314 9392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:22.110900 9392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:22.114629 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.112871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43571 (local address 127.0.0.1:46744)
0927 07:09:22.113185 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.113225 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:22.113281 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:22.113774 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:22.113780 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:22.113787 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:22.114204 (+ 417us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:22.114204 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:22.114218 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:22.114238 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:22.114461 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:22.114466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:22.114469 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:22.114535 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 07:09:22.114547 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":66,"threads_started":1}
I20260927 07:09:22.114748 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.112960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.4.174.66:43571)
0927 07:09:22.113373 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.113376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.113387 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:09:22.113413 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.113415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.113425 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.113508 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.114303 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.114311 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.114312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.114316 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.114350 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.114381 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.114436 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.114629 (+ 193us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.114646 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":96,"threads_started":1}
I20260927 07:09:22.114959 9851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.113065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:33781 (local address 127.0.0.1:42154)
0927 07:09:22.113354 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.113361 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:22.113401 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:22.113864 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:22.113865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:22.113868 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:22.114595 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:22.114596 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:22.114601 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:22.114609 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:22.114824 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:22.114825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:22.114825 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 07:09:22.114878 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 07:09:22.114900 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":63,"threads_started":1}
I20260927 07:09:22.115075 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.112871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:35205 (local address 127.0.0.1:36812)
0927 07:09:22.113184 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.113226 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:22.113281 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:22.113866 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:22.113866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:22.113869 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:22.114329 (+ 460us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:22.114330 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:22.114337 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:22.114351 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:22.114537 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:22.114537 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:22.114538 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 07:09:22.115014 (+ 476us) client_negotiation.cc:240] Negotiation successful
0927 07:09:22.115024 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":4,"thread_start_us":65,"threads_started":1}
I20260927 07:09:22.115142 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.113158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.4.174.67:33781)
0927 07:09:22.113315 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.113319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.113412 (+ 93us) server_negotiation.cc:407] Connection header received
0927 07:09:22.113435 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.113437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.113445 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.113508 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.114667 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.114675 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.114675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.114679 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.114713 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.114738 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.114797 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.115054 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.115069 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":106,"threads_started":1}
I20260927 07:09:22.115329 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.113039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36812 (local address 127.4.174.65:35205)
0927 07:09:22.113225 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.113230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.113439 (+ 209us) server_negotiation.cc:407] Connection header received
0927 07:09:22.113463 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.113465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.113473 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.113537 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.114421 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.114428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.114428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.114432 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.114460 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.114481 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.114537 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.115236 (+ 699us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.115251 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":65,"threads_started":1}
I20260927 07:09:22.115396 9645 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3ff6341e454fb8b063bfe397374988 (DEFAULT_TABLE table=test-table [id=b7bd23c6300d4ba2affda8bcd72c9856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:22.115770 9645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3ff6341e454fb8b063bfe397374988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:22.118150 9856 tablet_bootstrap.cc:492] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: Bootstrap starting.
I20260927 07:09:22.118721 9514 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3ff6341e454fb8b063bfe397374988 (DEFAULT_TABLE table=test-table [id=b7bd23c6300d4ba2affda8bcd72c9856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:22.118837 9856 tablet_bootstrap.cc:654] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:22.119030 9514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3ff6341e454fb8b063bfe397374988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:22.119067 9776 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3ff6341e454fb8b063bfe397374988 (DEFAULT_TABLE table=test-table [id=b7bd23c6300d4ba2affda8bcd72c9856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:22.119252 9856 log.cc:824] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:22.119387 9776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3ff6341e454fb8b063bfe397374988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:22.120036 9856 tablet_bootstrap.cc:492] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: No bootstrap required, opened a new log
I20260927 07:09:22.120110 9856 ts_tablet_manager.cc:1397] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:22.121871 9858 tablet_bootstrap.cc:492] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Bootstrap starting.
I20260927 07:09:22.122223 9859 tablet_bootstrap.cc:492] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf: Bootstrap starting.
I20260927 07:09:22.122509 9858 tablet_bootstrap.cc:654] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:22.122498 9856 raft_consensus.cc:348] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.122661 9856 raft_consensus.cc:374] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:22.122694 9856 raft_consensus.cc:729] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 487aa1fe3bac40c087b5506e941a68fd, State: Initialized, Role: FOLLOWER
I20260927 07:09:22.122784 9856 consensus_queue.cc:260] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.122821 9858 log.cc:824] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:22.122843 9859 tablet_bootstrap.cc:654] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:22.123011 9856 ts_tablet_manager.cc:1428] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:09:22.123027 9711 heartbeater.cc:503] Master 127.4.174.126:45955 was elected leader, sending a full tablet report...
I20260927 07:09:22.123142 9859 log.cc:824] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:22.123507 9858 tablet_bootstrap.cc:492] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: No bootstrap required, opened a new log
I20260927 07:09:22.123572 9858 ts_tablet_manager.cc:1397] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:22.123839 9859 tablet_bootstrap.cc:492] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf: No bootstrap required, opened a new log
I20260927 07:09:22.123896 9859 ts_tablet_manager.cc:1397] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 07:09:22.125491 9858 raft_consensus.cc:348] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.125604 9858 raft_consensus.cc:374] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:22.125622 9858 raft_consensus.cc:729] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3376110c9948bc8edec75cab991789, State: Initialized, Role: FOLLOWER
I20260927 07:09:22.125705 9858 consensus_queue.cc:260] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.125676 9859 raft_consensus.cc:348] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.125816 9859 raft_consensus.cc:374] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:22.125838 9859 raft_consensus.cc:729] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 507f4c54ca3f4b61b412ba1491c686bf, State: Initialized, Role: FOLLOWER
I20260927 07:09:22.125928 9858 ts_tablet_manager.cc:1428] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 07:09:22.125921 9859 consensus_queue.cc:260] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.125981 9842 heartbeater.cc:503] Master 127.4.174.126:45955 was elected leader, sending a full tablet report...
I20260927 07:09:22.126154 9859 ts_tablet_manager.cc:1428] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:09:22.126191 9580 heartbeater.cc:503] Master 127.4.174.126:45955 was elected leader, sending a full tablet report...
W20260927 07:09:22.126245 9581 tablet.cc:2369] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:09:22.239032 9712 tablet.cc:2369] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:22.330619 9863 raft_consensus.cc:482] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:22.330749 9863 raft_consensus.cc:504] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.331101 9863 leader_election.cc:288] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 507f4c54ca3f4b61b412ba1491c686bf (127.4.174.65:35205), 487aa1fe3bac40c087b5506e941a68fd (127.4.174.66:43571)
I20260927 07:09:22.332310 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.331292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43571 (local address 127.4.174.67:56623)
0927 07:09:22.331338 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.331350 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:22.331395 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:22.331611 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:22.331612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:22.331615 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:22.332003 (+ 388us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:22.332004 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:22.332011 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:22.332022 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:22.332183 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:22.332184 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:22.332185 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 07:09:22.332231 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 07:09:22.332243 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260927 07:09:22.332329 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.331450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:56623 (local address 127.4.174.66:43571)
0927 07:09:22.331504 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.331506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.331515 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:22.331542 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.331543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.331548 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.331631 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.332100 (+ 469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.332101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.332101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.332106 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.332124 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.332128 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.332180 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.332239 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.332253 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:22.332399 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.331308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:35205 (local address 127.4.174.67:57839)
0927 07:09:22.331485 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.331494 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:22.331539 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:22.331726 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:22.331727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:22.331729 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:22.332129 (+ 400us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:22.332130 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:22.332134 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:22.332143 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:22.332272 (+ 129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:22.332273 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:22.332273 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 07:09:22.332309 (+ 36us) client_negotiation.cc:240] Negotiation successful
0927 07:09:22.332332 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260927 07:09:22.332434 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.331442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:57839 (local address 127.4.174.65:35205)
0927 07:09:22.331502 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.331504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.331535 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:09:22.331636 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.331638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.331644 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.331705 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.332200 (+ 495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.332201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.332201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.332206 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.332221 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.332225 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.332275 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.332358 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.332372 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:09:22.332717 9665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3ff6341e454fb8b063bfe397374988" candidate_uuid: "da3376110c9948bc8edec75cab991789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487aa1fe3bac40c087b5506e941a68fd" is_pre_election: true
I20260927 07:09:22.332746 9534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3ff6341e454fb8b063bfe397374988" candidate_uuid: "da3376110c9948bc8edec75cab991789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "507f4c54ca3f4b61b412ba1491c686bf" is_pre_election: true
I20260927 07:09:22.332875 9665 raft_consensus.cc:2454] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3376110c9948bc8edec75cab991789 in term 0.
I20260927 07:09:22.332875 9534 raft_consensus.cc:2454] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3376110c9948bc8edec75cab991789 in term 0.
I20260927 07:09:22.333115 9729 leader_election.cc:302] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 487aa1fe3bac40c087b5506e941a68fd, da3376110c9948bc8edec75cab991789; no voters:
I20260927 07:09:22.333276 9863 raft_consensus.cc:2781] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:22.333335 9863 raft_consensus.cc:482] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:22.333351 9863 raft_consensus.cc:3037] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:22.334079 9863 raft_consensus.cc:504] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.334204 9863 leader_election.cc:288] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [CANDIDATE]: Term 1 election: Requested vote from peers 507f4c54ca3f4b61b412ba1491c686bf (127.4.174.65:35205), 487aa1fe3bac40c087b5506e941a68fd (127.4.174.66:43571)
I20260927 07:09:22.334331 9534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3ff6341e454fb8b063bfe397374988" candidate_uuid: "da3376110c9948bc8edec75cab991789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "507f4c54ca3f4b61b412ba1491c686bf"
I20260927 07:09:22.334357 9665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3ff6341e454fb8b063bfe397374988" candidate_uuid: "da3376110c9948bc8edec75cab991789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487aa1fe3bac40c087b5506e941a68fd"
I20260927 07:09:22.334414 9534 raft_consensus.cc:3037] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:22.334421 9665 raft_consensus.cc:3037] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:22.335304 9665 raft_consensus.cc:2454] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3376110c9948bc8edec75cab991789 in term 1.
I20260927 07:09:22.335304 9534 raft_consensus.cc:2454] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3376110c9948bc8edec75cab991789 in term 1.
I20260927 07:09:22.335464 9729 leader_election.cc:302] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 487aa1fe3bac40c087b5506e941a68fd, da3376110c9948bc8edec75cab991789; no voters:
I20260927 07:09:22.335541 9863 raft_consensus.cc:2781] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:22.335688 9863 raft_consensus.cc:686] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 1 LEADER]: Becoming Leader. State: Replica: da3376110c9948bc8edec75cab991789, State: Running, Role: LEADER
I20260927 07:09:22.335772 9863 consensus_queue.cc:237] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } }
I20260927 07:09:22.336587 9392 catalog_manager.cc:5495] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3376110c9948bc8edec75cab991789 (127.4.174.67). New cstate: current_term: 1 leader_uuid: "da3376110c9948bc8edec75cab991789" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3376110c9948bc8edec75cab991789" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33781 } health_report { overall_health: HEALTHY } } }
W20260927 07:09:22.348194 9843 tablet.cc:2369] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:22.395737 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.394778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42162 (local address 127.4.174.67:33781)
0927 07:09:22.394840 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.394842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.394862 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:09:22.394916 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.394917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.394932 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.394974 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.395527 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.395528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.395528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.395532 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.395548 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.395552 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.395599 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.395647 (+ 48us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.395660 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:22.397399 9534 raft_consensus.cc:1261] T 0a3ff6341e454fb8b063bfe397374988 P 507f4c54ca3f4b61b412ba1491c686bf [term 1 FOLLOWER]: Refusing update from remote peer da3376110c9948bc8edec75cab991789: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:22.397400 9665 raft_consensus.cc:1261] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 1 FOLLOWER]: Refusing update from remote peer da3376110c9948bc8edec75cab991789: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:22.397749 9866 consensus_queue.cc:1035] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [LEADER]: Connected to new peer: Peer: permanent_uuid: "507f4c54ca3f4b61b412ba1491c686bf" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:22.397869 9863 consensus_queue.cc:1035] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [LEADER]: Connected to new peer: Peer: permanent_uuid: "487aa1fe3bac40c087b5506e941a68fd" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:22.400110 9869 mvcc.cc:205] Tried to move back new op lower bound from 7333859173977374720 to 7333859173728006144. Current Snapshot: MvccSnapshot[applied={T|T < 7333859173977374720}]
I20260927 07:09:22.401175 9870 mvcc.cc:205] Tried to move back new op lower bound from 7333859173977374720 to 7333859173728006144. Current Snapshot: MvccSnapshot[applied={T|T < 7333859173977374720 or (T in {7333859173977374720})}]
I20260927 07:09:22.402664 9391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:22.402745 9391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:22.403416 9391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eefb61d70eeb4e7796e80ef90287377e: Sending DeleteTablet for 3 replicas of tablet 0a3ff6341e454fb8b063bfe397374988
I20260927 07:09:22.403738 9776 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3ff6341e454fb8b063bfe397374988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:22 UTC) from {username='slave'} at 127.0.0.1:42154
I20260927 07:09:22.403738 9645 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3ff6341e454fb8b063bfe397374988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:22 UTC) from {username='slave'} at 127.0.0.1:46744
I20260927 07:09:22.403749 9514 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3ff6341e454fb8b063bfe397374988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:22 UTC) from {username='slave'} at 127.0.0.1:36812
I20260927 07:09:22.403968 9879 tablet_replica.cc:330] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: stopping tablet replica
I20260927 07:09:22.404065 9879 raft_consensus.cc:2229] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:22.404105 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 9452
I20260927 07:09:22.404181 9879 raft_consensus.cc:2258] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:22.404309 9881 tablet_replica.cc:330] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: stopping tablet replica
I20260927 07:09:22.404392 9881 raft_consensus.cc:2229] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:22.404462 9881 raft_consensus.cc:2258] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:22.410861 9881 ts_tablet_manager.cc:1905] T 0a3ff6341e454fb8b063bfe397374988 P 487aa1fe3bac40c087b5506e941a68fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:22.410981 9879 ts_tablet_manager.cc:1905] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 07:09:22.411417 9378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:35205 (error 108)
I20260927 07:09:22.411549 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 9583
I20260927 07:09:22.412403 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.412281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:35205 (local address 127.0.0.1:36824)
0927 07:09:22.412327 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.412369 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:35205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260927 07:09:22.412634 9378 catalog_manager.cc:4584] TS 507f4c54ca3f4b61b412ba1491c686bf (127.4.174.65:35205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a3ff6341e454fb8b063bfe397374988: Network error: Client connection negotiation failed: client connection to 127.4.174.65:35205: connect: Connection refused (error 111)
I20260927 07:09:22.412859 9879 ts_tablet_manager.cc:1918] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:22.412920 9879 log.cc:1196] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/0a3ff6341e454fb8b063bfe397374988
I20260927 07:09:22.413162 9879 ts_tablet_manager.cc:1939] T 0a3ff6341e454fb8b063bfe397374988 P da3376110c9948bc8edec75cab991789: Deleting consensus metadata
I20260927 07:09:22.413653 9376 catalog_manager.cc:4841] TS da3376110c9948bc8edec75cab991789 (127.4.174.67:33781): tablet 0a3ff6341e454fb8b063bfe397374988 (table test-table [id=b7bd23c6300d4ba2affda8bcd72c9856]) successfully deleted
I20260927 07:09:22.416410 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.416295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43571 (local address 127.0.0.1:46754)
0927 07:09:22.416345 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.416375 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.66:43571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 07:09:22.416445 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 9714
I20260927 07:09:22.421342 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 9361
2026-09-27T07:09:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (809 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260927 07:09:22.439265 4793 mini_oidc.cc:152] Starting JWKS server
I20260927 07:09:22.440316 4793 webserver.cc:460] Webserver started at https://127.0.0.1:45779/ using document root <none> and password file <none>
I20260927 07:09:22.440423 4793 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 07:09:22.440584 4793 webserver.cc:460] Webserver started at http://127.0.0.1:34085/ using document root <none> and password file <none>
2026-09-27T07:09:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:22Z Disabled control of system clock
I20260927 07:09:22.447981 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35879
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:35879
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45779/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/testchainca.cert with env {}
W20260927 07:09:22.527525 9888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:22.527760 9888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:22.527782 9888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:22.528978 9888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:22.529026 9888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:22.529039 9888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:22.529066 9888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:22.529093 9888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 07:09:22.529119 9888 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 07:09:22.529145 9888 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45779/jwks.json
I20260927 07:09:22.530966 9888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:35879
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45779/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:22.531234 9888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:22.531431 9888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:22.535962 9893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:22.536044 9888 server_base.cc:1034] running on GCE node
W20260927 07:09:22.535993 9894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:22.535982 9896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:22.536382 9888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:22.536674 9888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:22.537819 9888 hybrid_clock.cc:648] HybridClock initialized: now 1790492962537803 us; error 29 us; skew 500 ppm
I20260927 07:09:22.539062 9888 webserver.cc:460] Webserver started at http://127.4.174.126:46109/ using document root <none> and password file <none>
I20260927 07:09:22.539278 9888 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:22.539327 9888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:22.539451 9888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:22.540530 9888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "0c9a30feebf84a53b1325529a2e1b6d1"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "c6bbd2707dcd42ddc656c25e56186c0c"
server_key_iv: "36cc61182cf3462d94144252ce50fc8d"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.540866 9888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "0c9a30feebf84a53b1325529a2e1b6d1"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "c6bbd2707dcd42ddc656c25e56186c0c"
server_key_iv: "36cc61182cf3462d94144252ce50fc8d"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.542181 9888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:22.542914 9902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:22.543066 9888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:22.543147 9888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "0c9a30feebf84a53b1325529a2e1b6d1"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "c6bbd2707dcd42ddc656c25e56186c0c"
server_key_iv: "36cc61182cf3462d94144252ce50fc8d"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.543206 9888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:22.555507 9888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:22.555830 9888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:22.555958 9888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:22.559556 9888 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:35879
I20260927 07:09:22.559603 9954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:35879 every 8 connection(s)
I20260927 07:09:22.559935 9888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:22.560846 9955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:22.561911 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 9888
I20260927 07:09:22.562005 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:22.562168 4793 external_mini_cluster.cc:1428] Setting key ec91f85a57e768f7ec7ce8747c324626
I20260927 07:09:22.563329 9955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1: Bootstrap starting.
I20260927 07:09:22.563917 9955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:22.564178 9955 log.cc:824] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:22.564775 9955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1: No bootstrap required, opened a new log
I20260927 07:09:22.566644 9955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9a30feebf84a53b1325529a2e1b6d1" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35879 } }
I20260927 07:09:22.566828 9955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:22.566871 9955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9a30feebf84a53b1325529a2e1b6d1, State: Initialized, Role: FOLLOWER
I20260927 07:09:22.567003 9955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9a30feebf84a53b1325529a2e1b6d1" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35879 } }
I20260927 07:09:22.567087 9955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:22.567117 9955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:22.567205 9955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:22.567411 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.562668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.4.174.126:35879)
0927 07:09:22.562838 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.562846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.562871 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:09:22.563272 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.563278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.563345 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.563477 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.563679 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.564578 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.565391 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.565806 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.565914 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:22.566680 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.566685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.566685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.566896 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.566940 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.566955 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.567022 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.567099 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.567185 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":55,"threads_started":1}
I20260927 07:09:22.567884 9955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9a30feebf84a53b1325529a2e1b6d1" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35879 } }
I20260927 07:09:22.567991 9955 leader_election.cc:302] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c9a30feebf84a53b1325529a2e1b6d1; no voters:
I20260927 07:09:22.568151 9955 leader_election.cc:288] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:22.568274 9960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:22.568373 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:09:22.568425 9955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:22.569074 9960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [term 1 LEADER]: Becoming Leader. State: Replica: 0c9a30feebf84a53b1325529a2e1b6d1, State: Running, Role: LEADER
I20260927 07:09:22.569327 9960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9a30feebf84a53b1325529a2e1b6d1" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35879 } }
I20260927 07:09:22.569695 9960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c9a30feebf84a53b1325529a2e1b6d1. Latest consensus state: current_term: 1 leader_uuid: "0c9a30feebf84a53b1325529a2e1b6d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9a30feebf84a53b1325529a2e1b6d1" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35879 } } }
I20260927 07:09:22.569765 9960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:22.569897 9961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c9a30feebf84a53b1325529a2e1b6d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9a30feebf84a53b1325529a2e1b6d1" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35879 } } }
I20260927 07:09:22.569981 9961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1 [sys.catalog]: This master's current role is: LEADER
W20260927 07:09:22.570752 9975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:09:22.570807 9975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:09:22.570863 9969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:22.571048 9969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:22.572345 9969 catalog_manager.cc:1347] Generated new cluster ID: 0d1ad59bf0b7480cbdb68979878b1332
I20260927 07:09:22.572404 9969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:22.588315 9969 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:22.588841 9969 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:22.590911 9969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c9a30feebf84a53b1325529a2e1b6d1: Generated new TSK 0
I20260927 07:09:22.591089 9969 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 07:09:22.623101 9985 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260927 07:09:22.623214 9985 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 07:09:22.623386 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.600844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.4.174.126:35879)
0927 07:09:22.600895 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.600898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.600995 (+ 97us) server_negotiation.cc:407] Connection header received
0927 07:09:22.601100 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.601102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.601113 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.601168 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0927 07:09:22.601312 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.601814 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.602586 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.602789 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.602844 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:22.603051 (+ 207us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 07:09:22.623138 (+ 20087us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 07:09:22.623301 (+ 163us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56262: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260927 07:09:22.623472 9958 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56262: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260927 07:09:22.623899 9984 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:22.600734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35879 (local address 127.0.0.1:56262)
0927 07:09:22.600933 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.600944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:22.601009 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:22.601185 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:22.601186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:22.601188 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 07:09:22.601258 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:22.601261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.601904 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.601905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:22.602495 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:22.602497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.602850 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.602851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:22.602927 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:22.602950 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 07:09:22.623424 (+ 20474us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 07:09:22.623555 (+ 131us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.174.126:35879: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.174.126:35879: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260927 07:09:22.624333 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 9888
2026-09-27T07:09:22Z chronyd exiting
I20260927 07:09:22.659417 4793 test_util.cc:175] -----------------------------------------------
I20260927 07:09:22.659498 4793 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492940996973-4793-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (221 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260927 07:09:22.660245 4793 mini_oidc.cc:152] Starting JWKS server
I20260927 07:09:22.660907 4793 webserver.cc:460] Webserver started at https://127.0.0.1:33715/ using document root <none> and password file <none>
I20260927 07:09:22.660949 4793 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 07:09:22.661077 4793 webserver.cc:460] Webserver started at http://127.0.0.1:37845/ using document root <none> and password file <none>
2026-09-27T07:09:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:22Z Disabled control of system clock
I20260927 07:09:22.667884 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:42107
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:40787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:42107
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33715/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/testchainca.cert with env {}
W20260927 07:09:22.745963 9991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:22.746183 9991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:22.746212 9991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:22.747401 9991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:22.747444 9991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:22.747460 9991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:22.747476 9991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:22.747491 9991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 07:09:22.747507 9991 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 07:09:22.747522 9991 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33715/jwks.json
I20260927 07:09:22.749032 9991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:42107
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:42107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33715/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:22.749277 9991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:22.749475 9991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:22.754034 9997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:22.754258 9991 server_base.cc:1034] running on GCE node
W20260927 07:09:22.754040 9996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:22.754052 9999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:22.754618 9991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:22.754899 9991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:22.756034 9991 hybrid_clock.cc:648] HybridClock initialized: now 1790492962756017 us; error 28 us; skew 500 ppm
I20260927 07:09:22.757123 9991 webserver.cc:460] Webserver started at http://127.4.174.126:37637/ using document root <none> and password file <none>
I20260927 07:09:22.757357 9991 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:22.757404 9991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:22.757527 9991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:22.758530 9991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "8e5abe778c0d43e5a5ad83bf0f06f953"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "d8264f27e703527bd56b2ffe888a23af"
server_key_iv: "a3d5a3d979bd75589ecf59521450535c"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.758908 9991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "8e5abe778c0d43e5a5ad83bf0f06f953"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "d8264f27e703527bd56b2ffe888a23af"
server_key_iv: "a3d5a3d979bd75589ecf59521450535c"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.760085 9991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:22.760766 10005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:22.760911 9991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:22.760990 9991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "8e5abe778c0d43e5a5ad83bf0f06f953"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "d8264f27e703527bd56b2ffe888a23af"
server_key_iv: "a3d5a3d979bd75589ecf59521450535c"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.761056 9991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:22.782086 9991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:22.782407 9991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:22.782536 9991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:22.786065 9991 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:42107
I20260927 07:09:22.786089 10057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:42107 every 8 connection(s)
I20260927 07:09:22.786451 9991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:22.787427 10058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:22.789923 10058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953: Bootstrap starting.
I20260927 07:09:22.790504 10058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:22.790791 10058 log.cc:824] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:22.791479 10058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953: No bootstrap required, opened a new log
I20260927 07:09:22.791550 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 9991
I20260927 07:09:22.791651 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:22.791832 4793 external_mini_cluster.cc:1428] Setting key f20c650dcd297851ff4105d4a2a00985
I20260927 07:09:22.793695 10058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5abe778c0d43e5a5ad83bf0f06f953" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42107 } }
I20260927 07:09:22.793884 10058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:22.793924 10058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e5abe778c0d43e5a5ad83bf0f06f953, State: Initialized, Role: FOLLOWER
I20260927 07:09:22.794082 10058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5abe778c0d43e5a5ad83bf0f06f953" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42107 } }
I20260927 07:09:22.794188 10058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:22.794232 10058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:22.794297 10058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:22.794938 10058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5abe778c0d43e5a5ad83bf0f06f953" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42107 } }
I20260927 07:09:22.795049 10058 leader_election.cc:302] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e5abe778c0d43e5a5ad83bf0f06f953; no voters:
I20260927 07:09:22.795202 10058 leader_election.cc:288] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:22.795226 10063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:22.795439 10063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [term 1 LEADER]: Becoming Leader. State: Replica: 8e5abe778c0d43e5a5ad83bf0f06f953, State: Running, Role: LEADER
I20260927 07:09:22.795526 10058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:22.795574 10063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5abe778c0d43e5a5ad83bf0f06f953" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42107 } }
I20260927 07:09:22.795992 10063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e5abe778c0d43e5a5ad83bf0f06f953. Latest consensus state: current_term: 1 leader_uuid: "8e5abe778c0d43e5a5ad83bf0f06f953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5abe778c0d43e5a5ad83bf0f06f953" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42107 } } }
I20260927 07:09:22.796054 10063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:22.796061 10064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e5abe778c0d43e5a5ad83bf0f06f953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5abe778c0d43e5a5ad83bf0f06f953" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42107 } } }
I20260927 07:09:22.796145 10064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:22.796372 10070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:22.797259 10070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:22.798960 10070 catalog_manager.cc:1347] Generated new cluster ID: 067a4a25e4664b3599751725bf37d2c9
I20260927 07:09:22.799010 10070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:22.799624 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.792850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45594 (local address 127.4.174.126:42107)
0927 07:09:22.793034 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.793041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.793065 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:09:22.793655 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.793665 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.793779 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.793937 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:22.794389 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.796236 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.797380 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.797968 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.798092 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:22.798801 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:22.798805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:22.798805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:22.799003 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:22.799045 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:22.799061 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:22.799120 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:22.799316 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 07:09:22.799408 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260927 07:09:22.803568 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:09:22.808411 10070 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:22.808900 10070 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:22.811352 10070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e5abe778c0d43e5a5ad83bf0f06f953: Generated new TSK 0
I20260927 07:09:22.811513 10070 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 07:09:22.833871 10088 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260927 07:09:22.833950 10088 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 07:09:22.834285 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:22.819851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45602 (local address 127.4.174.126:42107)
0927 07:09:22.819918 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:22.819920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:22.820006 (+ 86us) server_negotiation.cc:407] Connection header received
0927 07:09:22.820064 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:22.820065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:22.820077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:22.820155 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0927 07:09:22.820267 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.821085 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.821855 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.822185 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.822251 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:22.822449 (+ 198us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 07:09:22.834049 (+ 11600us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 07:09:22.834205 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":42}
W20260927 07:09:22.834347 10062 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260927 07:09:22.834378 10087 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:22.819787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42107 (local address 127.0.0.1:45602)
0927 07:09:22.819946 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:22.819955 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:22.820009 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:22.820153 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:22.820154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:22.820156 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 07:09:22.820206 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:22.820207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.821177 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.821178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:22.821785 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:22.821787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:22.822277 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:22.822278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:22.822348 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:22.822354 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 07:09:22.834159 (+ 11805us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 07:09:22.834291 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.174.126:42107: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260927 07:09:22.834812 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 9991
2026-09-27T07:09:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260927 07:09:22.871591 4793 mini_oidc.cc:152] Starting JWKS server
I20260927 07:09:22.872277 4793 webserver.cc:460] Webserver started at https://127.0.0.1:37065/ using document root <none> and password file <none>
I20260927 07:09:22.872323 4793 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 07:09:22.872457 4793 webserver.cc:460] Webserver started at http://127.0.0.1:40685/ using document root <none> and password file <none>
2026-09-27T07:09:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:22Z Disabled control of system clock
I20260927 07:09:22.879704 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:34961
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:34961
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37065/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 07:09:22.959527 10094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:22.959726 10094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:22.959746 10094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:22.960870 10094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:22.960904 10094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:22.960915 10094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:22.960927 10094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:22.960937 10094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 07:09:22.960950 10094 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 07:09:22.960960 10094 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37065/jwks.json
I20260927 07:09:22.962280 10094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:34961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37065/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:22.962466 10094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:22.962731 10094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:22.967133 10100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:22.967130 10102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:22.967226 10094 server_base.cc:1034] running on GCE node
W20260927 07:09:22.967139 10099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:22.967584 10094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:22.967842 10094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:22.968983 10094 hybrid_clock.cc:648] HybridClock initialized: now 1790492962968968 us; error 28 us; skew 500 ppm
I20260927 07:09:22.970155 10094 webserver.cc:460] Webserver started at http://127.4.174.126:45875/ using document root <none> and password file <none>
I20260927 07:09:22.970386 10094 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:22.970435 10094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:22.970582 10094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:22.971601 10094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "b5c57091b4474e068d5018d8017e2857"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "60213beafe34167fb8f60c77934a2f3f"
server_key_iv: "d821974fed6777961d6cabbd8a748622"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.971928 10094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "b5c57091b4474e068d5018d8017e2857"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "60213beafe34167fb8f60c77934a2f3f"
server_key_iv: "d821974fed6777961d6cabbd8a748622"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.973125 10094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:22.973747 10108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:22.973909 10094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:22.973991 10094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "b5c57091b4474e068d5018d8017e2857"
format_stamp: "Formatted at 2026-09-27 07:09:22 on dist-test-slave-zw7p"
server_key: "60213beafe34167fb8f60c77934a2f3f"
server_key_iv: "d821974fed6777961d6cabbd8a748622"
server_key_version: "encryptionkey@0"
I20260927 07:09:22.974057 10094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:22.990417 10094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:22.990741 10094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:22.990871 10094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:22.994522 10094 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:34961
I20260927 07:09:22.994580 10160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:34961 every 8 connection(s)
I20260927 07:09:22.994946 10094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:22.995569 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:22.998194 10161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857: Bootstrap starting.
I20260927 07:09:22.998735 10161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:22.998967 10161 log.cc:824] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:22.999573 10161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857: No bootstrap required, opened a new log
I20260927 07:09:23.001111 10161 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c57091b4474e068d5018d8017e2857" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34961 } }
I20260927 07:09:23.001258 10161 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:23.001281 10161 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5c57091b4474e068d5018d8017e2857, State: Initialized, Role: FOLLOWER
I20260927 07:09:23.001394 10161 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c57091b4474e068d5018d8017e2857" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34961 } }
I20260927 07:09:23.001470 10161 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:23.001492 10161 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:23.001521 10161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:23.002023 10161 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c57091b4474e068d5018d8017e2857" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34961 } }
I20260927 07:09:23.002110 10161 leader_election.cc:302] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5c57091b4474e068d5018d8017e2857; no voters:
I20260927 07:09:23.002235 10161 leader_election.cc:288] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:23.002293 10164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:23.002465 10161 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:23.002485 10164 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [term 1 LEADER]: Becoming Leader. State: Replica: b5c57091b4474e068d5018d8017e2857, State: Running, Role: LEADER
I20260927 07:09:23.002681 10164 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c57091b4474e068d5018d8017e2857" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34961 } }
I20260927 07:09:23.003053 10165 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5c57091b4474e068d5018d8017e2857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c57091b4474e068d5018d8017e2857" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34961 } } }
I20260927 07:09:23.003131 10165 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:23.003319 10173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:23.003522 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 10094
I20260927 07:09:23.003615 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:23.003633 10166 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5c57091b4474e068d5018d8017e2857. Latest consensus state: current_term: 1 leader_uuid: "b5c57091b4474e068d5018d8017e2857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c57091b4474e068d5018d8017e2857" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34961 } } }
I20260927 07:09:23.003705 10166 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:23.003798 4793 external_mini_cluster.cc:1428] Setting key 4a0b11c0d41e3c5592dc265db9600515
I20260927 07:09:23.003952 10173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:23.005465 10173 catalog_manager.cc:1347] Generated new cluster ID: 5431538c38f84824825c6b97bcdd82a5
I20260927 07:09:23.005507 10173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:23.009733 10183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:23.004379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36010 (local address 127.4.174.126:34961)
0927 07:09:23.004573 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:23.004581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:23.004614 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:09:23.004747 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:23.004755 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:23.004824 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:23.004955 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:23.005159 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.006492 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.007597 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.008262 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.008394 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:23.009027 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:23.009031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:23.009031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:23.009241 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:23.009285 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:23.009300 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:23.009381 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:23.009440 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 07:09:23.009522 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260927 07:09:23.010663 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:09:23.032603 10173 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:23.033113 10173 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:23.037658 10173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5c57091b4474e068d5018d8017e2857: Generated new TSK 0
I20260927 07:09:23.037839 10173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:09:23.052775 10183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:23.048815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36018 (local address 127.4.174.126:34961)
0927 07:09:23.048866 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:23.048868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:23.048934 (+ 66us) server_negotiation.cc:407] Connection header received
0927 07:09:23.048999 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:23.049000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:23.049013 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:23.049103 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0927 07:09:23.049279 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.050079 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.050960 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.051280 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.051357 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:23.051525 (+ 168us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 07:09:23.052536 (+ 1011us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 07:09:23.052677 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36018: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260927 07:09:23.052954 10183 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36018: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260927 07:09:23.054829 10190 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:23.048704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:34961 (local address 127.0.0.1:36018)
0927 07:09:23.048867 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:23.048877 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:23.048934 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:23.049138 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:23.049139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:23.049142 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0927 07:09:23.049207 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:23.049209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.050198 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.050199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:23.050886 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:23.050891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.051361 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.051362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:23.051420 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:23.051426 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 07:09:23.052666 (+ 1240us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 07:09:23.054726 (+ 2060us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.174.126:34961: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":1838,"thread_start_us":98,"threads_started":1}
I20260927 07:09:23.055092 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 10094
E20260927 07:09:23.063635 10191 webserver.cc:533] Webserver: sslize failed:
E20260927 07:09:23.063707 10191 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-27T07:09:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (220 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-27T07:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:23Z Disabled control of system clock
I20260927 07:09:23.098861 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:46593
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:46593
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/testchainca.cert with env {}
W20260927 07:09:23.176590 10195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:23.176798 10195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:23.176828 10195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:23.177997 10195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:23.178038 10195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:23.178053 10195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:23.178068 10195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:23.178083 10195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 07:09:23.178099 10195 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 07:09:23.178118 10195 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260927 07:09:23.179632 10195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:46593
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:46593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:23.179893 10195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:23.180094 10195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:23.184621 10201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:23.184639 10203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:23.184733 10200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:23.185032 10195 server_base.cc:1034] running on GCE node
I20260927 07:09:23.185245 10195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:23.185510 10195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:23.186606 10195 hybrid_clock.cc:648] HybridClock initialized: now 1790492963186590 us; error 39 us; skew 500 ppm
I20260927 07:09:23.187737 10195 webserver.cc:460] Webserver started at http://127.4.174.126:43901/ using document root <none> and password file <none>
I20260927 07:09:23.187974 10195 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:23.188020 10195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:23.188138 10195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:23.189147 10195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "e0bebec990fe4b0e842ca78dad7d0090"
format_stamp: "Formatted at 2026-09-27 07:09:23 on dist-test-slave-zw7p"
server_key: "d2c9693bebfadf2c6042f5c932b14acd"
server_key_iv: "c1d20fd1f1343f3a6dd5bf12b87a5162"
server_key_version: "encryptionkey@0"
I20260927 07:09:23.189456 10195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "e0bebec990fe4b0e842ca78dad7d0090"
format_stamp: "Formatted at 2026-09-27 07:09:23 on dist-test-slave-zw7p"
server_key: "d2c9693bebfadf2c6042f5c932b14acd"
server_key_iv: "c1d20fd1f1343f3a6dd5bf12b87a5162"
server_key_version: "encryptionkey@0"
I20260927 07:09:23.190665 10195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:23.191408 10209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:23.191545 10195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:23.191618 10195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "e0bebec990fe4b0e842ca78dad7d0090"
format_stamp: "Formatted at 2026-09-27 07:09:23 on dist-test-slave-zw7p"
server_key: "d2c9693bebfadf2c6042f5c932b14acd"
server_key_iv: "c1d20fd1f1343f3a6dd5bf12b87a5162"
server_key_version: "encryptionkey@0"
I20260927 07:09:23.191679 10195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:23.204401 10195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:23.204703 10195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:23.204830 10195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:23.208710 10195 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:46593
I20260927 07:09:23.208768 10261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:46593 every 8 connection(s)
I20260927 07:09:23.209098 10195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:23.210031 10262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:23.212356 10262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090: Bootstrap starting.
I20260927 07:09:23.212800 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 10195
I20260927 07:09:23.212898 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:23.213001 10262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:23.213088 4793 external_mini_cluster.cc:1428] Setting key f8e34311c1d0f5064a68dfe3189b60e7
I20260927 07:09:23.213310 10262 log.cc:824] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:23.213973 10262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090: No bootstrap required, opened a new log
I20260927 07:09:23.216301 10262 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bebec990fe4b0e842ca78dad7d0090" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46593 } }
I20260927 07:09:23.216439 10262 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:23.216459 10262 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0bebec990fe4b0e842ca78dad7d0090, State: Initialized, Role: FOLLOWER
I20260927 07:09:23.216575 10262 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bebec990fe4b0e842ca78dad7d0090" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46593 } }
I20260927 07:09:23.216652 10262 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:23.216681 10262 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:23.216724 10262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:23.217382 10262 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bebec990fe4b0e842ca78dad7d0090" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46593 } }
I20260927 07:09:23.217490 10262 leader_election.cc:302] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0bebec990fe4b0e842ca78dad7d0090; no voters:
I20260927 07:09:23.217629 10262 leader_election.cc:288] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:23.217660 10267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:23.217860 10267 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [term 1 LEADER]: Becoming Leader. State: Replica: e0bebec990fe4b0e842ca78dad7d0090, State: Running, Role: LEADER
I20260927 07:09:23.217875 10262 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:23.218019 10267 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bebec990fe4b0e842ca78dad7d0090" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46593 } }
I20260927 07:09:23.218398 10268 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0bebec990fe4b0e842ca78dad7d0090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bebec990fe4b0e842ca78dad7d0090" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46593 } } }
I20260927 07:09:23.218474 10268 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:23.218695 10269 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0bebec990fe4b0e842ca78dad7d0090. Latest consensus state: current_term: 1 leader_uuid: "e0bebec990fe4b0e842ca78dad7d0090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bebec990fe4b0e842ca78dad7d0090" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46593 } } }
I20260927 07:09:23.218803 10269 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:23.218967 10275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:23.219484 10275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:23.219738 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:23.213573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57630 (local address 127.4.174.126:46593)
0927 07:09:23.213775 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:23.213785 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:23.213813 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:09:23.215116 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:23.215123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:23.215185 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:23.215300 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:23.215522 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.216403 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.217464 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.218074 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.218224 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:23.218975 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:23.218980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:23.218980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:23.219193 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:23.219239 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:23.219256 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:23.219314 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:23.219437 (+ 123us) server_negotiation.cc:299] Negotiation successful
0927 07:09:23.219532 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260927 07:09:23.220739 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:09:23.222950 10275 catalog_manager.cc:1347] Generated new cluster ID: bef7547514ad46ca9eb80be35c376a26
I20260927 07:09:23.223011 10275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:23.240449 10275 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:23.240985 10275 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:23.245609 10275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0bebec990fe4b0e842ca78dad7d0090: Generated new TSK 0
I20260927 07:09:23.245806 10275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:09:23.723489 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:23.259346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.4.174.126:46593)
0927 07:09:23.259399 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:23.259401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:23.259503 (+ 102us) server_negotiation.cc:407] Connection header received
0927 07:09:23.259546 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:23.259547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:23.259560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:23.259640 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0927 07:09:23.259747 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.260555 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.261358 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.261683 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.261750 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:23.262036 (+ 286us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 07:09:23.723237 (+461201us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 07:09:23.723429 (+ 192us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57634: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260927 07:09:23.723551 10266 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57634: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260927 07:09:23.723600 10291 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:23.259269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:46593 (local address 127.0.0.1:57634)
0927 07:09:23.259435 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:23.259443 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:23.259494 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:23.259636 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:23.259637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:23.259638 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0927 07:09:23.259693 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:23.259694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.260640 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.260641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:23.261273 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:23.261275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:23.261800 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:23.261802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:23.261880 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:23.261887 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 07:09:23.723398 (+461511us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 07:09:23.723514 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.174.126:46593: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260927 07:09:23.724015 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 10195
2026-09-27T07:09:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (648 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:23 dist-test-slave-zw7p krb5kdc[10303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:23 dist-test-slave-zw7p krb5kdc[10303](info): set up 2 sockets
Sep 27 07:09:23 dist-test-slave-zw7p krb5kdc[10303](info): commencing operation
krb5kdc: starting...
W20260927 07:09:24.846433 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:24 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:24.957537 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:45187
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:45187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:25.035213 10316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:25.035418 10316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:25.035449 10316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:25.036602 10316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:25.036640 10316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:25.036659 10316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:25.036675 10316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:25.036691 10316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:25.038184 10316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:45187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:45187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:25.038414 10316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:25.038626 10316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:25.043260 10321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:25.043267 10322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:25.043409 10316 server_base.cc:1034] running on GCE node
W20260927 07:09:25.043322 10324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:25.043725 10316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:25.044025 10316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:25.045194 10316 hybrid_clock.cc:648] HybridClock initialized: now 1790492965045177 us; error 28 us; skew 500 ppm
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.052440 10316 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:25.052893 10316 webserver.cc:460] Webserver started at http://127.4.174.126:43309/ using document root <none> and password file <none>
I20260927 07:09:25.053121 10316 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:25.053174 10316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:25.053310 10316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:25.054378 10316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "f04c9e86159e41bf95b7cb6344b61e7c"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "f536c3505d0cc98c52f8f907eb96b930"
server_key_iv: "82cc218c886da7efe12535a8394edabb"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.054759 10316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "f04c9e86159e41bf95b7cb6344b61e7c"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "f536c3505d0cc98c52f8f907eb96b930"
server_key_iv: "82cc218c886da7efe12535a8394edabb"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.056058 10316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:25.056813 10331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.056977 10316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:25.057061 10316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "f04c9e86159e41bf95b7cb6344b61e7c"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "f536c3505d0cc98c52f8f907eb96b930"
server_key_iv: "82cc218c886da7efe12535a8394edabb"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.057157 10316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:25.080428 10316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:25.082365 10316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:25.082499 10316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:25.086195 10316 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:45187
I20260927 07:09:25.086241 10383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:45187 every 8 connection(s)
I20260927 07:09:25.086603 10316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:25.087414 10384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:25.089557 10384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c: Bootstrap starting.
I20260927 07:09:25.090052 10384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:25.090282 10384 log.cc:824] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:25.090932 10384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c: No bootstrap required, opened a new log
I20260927 07:09:25.092136 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 10316
I20260927 07:09:25.092226 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:25.092411 4793 external_mini_cluster.cc:1428] Setting key df1ce97a7726e3a678d2d32dc1bc931a
I20260927 07:09:25.092469 10384 raft_consensus.cc:348] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04c9e86159e41bf95b7cb6344b61e7c" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45187 } }
I20260927 07:09:25.092622 10384 raft_consensus.cc:374] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:25.092659 10384 raft_consensus.cc:729] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f04c9e86159e41bf95b7cb6344b61e7c, State: Initialized, Role: FOLLOWER
I20260927 07:09:25.092808 10384 consensus_queue.cc:260] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04c9e86159e41bf95b7cb6344b61e7c" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45187 } }
I20260927 07:09:25.092886 10384 raft_consensus.cc:388] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:25.092916 10384 raft_consensus.cc:482] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:25.092959 10384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:25.093465 10384 raft_consensus.cc:504] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04c9e86159e41bf95b7cb6344b61e7c" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45187 } }
I20260927 07:09:25.093560 10384 leader_election.cc:302] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f04c9e86159e41bf95b7cb6344b61e7c; no voters:
I20260927 07:09:25.093707 10384 leader_election.cc:288] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:25.093752 10389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:25.093850 10389 raft_consensus.cc:686] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [term 1 LEADER]: Becoming Leader. State: Replica: f04c9e86159e41bf95b7cb6344b61e7c, State: Running, Role: LEADER
I20260927 07:09:25.093930 10389 consensus_queue.cc:237] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04c9e86159e41bf95b7cb6344b61e7c" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45187 } }
I20260927 07:09:25.093963 10384 sys_catalog.cc:564] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:25.094269 10389 sys_catalog.cc:455] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f04c9e86159e41bf95b7cb6344b61e7c. Latest consensus state: current_term: 1 leader_uuid: "f04c9e86159e41bf95b7cb6344b61e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04c9e86159e41bf95b7cb6344b61e7c" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45187 } } }
I20260927 07:09:25.094334 10389 sys_catalog.cc:458] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:25.094579 10391 sys_catalog.cc:455] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f04c9e86159e41bf95b7cb6344b61e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04c9e86159e41bf95b7cb6344b61e7c" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45187 } } }
I20260927 07:09:25.094709 10391 sys_catalog.cc:458] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:25.094707 10394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:25.095235 10394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:25.096661 10394 catalog_manager.cc:1347] Generated new cluster ID: 325cac93ad8b4a2eb3d4727c0bf8806b
I20260927 07:09:25.096710 10394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:25.104111 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.092876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.4.174.126:45187)
0927 07:09:25.093065 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.093075 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.093116 (+ 41us) server_negotiation.cc:407] Connection header received
0927 07:09:25.093429 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.093435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.093479 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.093599 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.094383 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.095945 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.096952 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.097548 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.097655 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.100376 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:25.100381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:25.100382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:25.100394 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:25.102372 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.102763 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.102767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.102767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.102897 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.103072 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.103073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.103073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.103403 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:25.103565 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:25.103647 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 07:09:25.103881 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260927 07:09:25.120131 10394 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:25.120666 10394 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:25.126148 10394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f04c9e86159e41bf95b7cb6344b61e7c: Generated new TSK 0
I20260927 07:09:25.126317 10394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:25.153903 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45187
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:25.233048 10410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:25.233263 10410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:25.233286 10410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:25.234431 10410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:25.234469 10410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:25.234493 10410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:25.235963 10410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:25.236179 10410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:25.236406 10410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:25.241323 10415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:25.241329 10418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:25.241336 10416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:25.241529 10410 server_base.cc:1034] running on GCE node
I20260927 07:09:25.241806 10410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:25.242031 10410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:25.243187 10410 hybrid_clock.cc:648] HybridClock initialized: now 1790492965243171 us; error 29 us; skew 500 ppm
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.245635 10410 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:25.246048 10410 webserver.cc:460] Webserver started at http://127.4.174.65:36293/ using document root <none> and password file <none>
I20260927 07:09:25.246270 10410 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:25.246317 10410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:25.246423 10410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:25.247520 10410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "faf37e5506244c16873f75815f8ceefd"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "6fe82d9687d84a6e9aab05e48a23b3ad"
server_key_iv: "abcf5e87a5bae43e3abb3dbd875019dc"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.247848 10410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "faf37e5506244c16873f75815f8ceefd"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "6fe82d9687d84a6e9aab05e48a23b3ad"
server_key_iv: "abcf5e87a5bae43e3abb3dbd875019dc"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.249064 10410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:25.249817 10425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.249972 10410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:25.250048 10410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "faf37e5506244c16873f75815f8ceefd"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "6fe82d9687d84a6e9aab05e48a23b3ad"
server_key_iv: "abcf5e87a5bae43e3abb3dbd875019dc"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.250115 10410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:25.268363 10410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:25.270109 10410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:25.270229 10410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:25.270455 10410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:25.270799 10410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:25.270833 10410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.270854 10410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:25.270867 10410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.276255 10410 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:44127
I20260927 07:09:25.276315 10538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:44127 every 8 connection(s)
I20260927 07:09:25.276618 10410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:25.278540 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 10410
I20260927 07:09:25.278657 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:25.278846 4793 external_mini_cluster.cc:1428] Setting key 45c207bcadf26044b0812fcea0099987
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
I20260927 07:09:25.285836 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.277317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:36005 (local address 127.4.174.126:45187)
0927 07:09:25.277367 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.277369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.277606 (+ 237us) server_negotiation.cc:407] Connection header received
0927 07:09:25.278730 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.278732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.278743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.278797 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.279567 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.280096 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.281085 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.281281 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.281336 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.283351 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:25.283352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:25.283353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:25.283361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:25.284593 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.284957 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.284958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.284959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.285010 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.285219 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.285220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.285220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.285327 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:25.285382 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:25.285657 (+ 275us) server_negotiation.cc:299] Negotiation successful
0927 07:09:25.285746 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:09:25.285929 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.277253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45187 (local address 127.4.174.65:36005)
0927 07:09:25.277493 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:25.277532 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:25.278586 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:25.278879 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:25.278883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:25.279222 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:25.279490 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:25.279497 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.280187 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.280188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:25.281003 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:25.281006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.281348 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.281349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:25.281487 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.281954 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:25.281962 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:25.283264 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:25.284669 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:25.284676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:25.284679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:25.284890 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:25.285063 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:25.285063 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:25.285063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:25.285163 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:25.285381 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:25.285387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:25.285517 (+ 130us) client_negotiation.cc:769] Sending connection context
0927 07:09:25.285612 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 07:09:25.285758 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":72,"threads_started":1}
I20260927 07:09:25.286334 10539 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45187
I20260927 07:09:25.286432 10539 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:25.286628 10539 heartbeater.cc:511] Master 127.4.174.126:45187 requested a full tablet report, sending...
I20260927 07:09:25.287209 10348 ts_manager.cc:194] Registered new tserver with Master: faf37e5506244c16873f75815f8ceefd (127.4.174.65:44127)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
I20260927 07:09:25.287832 10348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:36005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:25.301440 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45187
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:25.379688 10544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:25.379895 10544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:25.379917 10544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:25.381044 10544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:25.381086 10544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:25.381111 10544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:25.382539 10544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:25.382779 10544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:25.382985 10544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:25.387677 10550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:25.387701 10552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:25.387753 10549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:25.387949 10544 server_base.cc:1034] running on GCE node
I20260927 07:09:25.388135 10544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:25.388387 10544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:25.389511 10544 hybrid_clock.cc:648] HybridClock initialized: now 1790492965389499 us; error 33 us; skew 500 ppm
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.391884 10544 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:25.392310 10544 webserver.cc:460] Webserver started at http://127.4.174.66:45163/ using document root <none> and password file <none>
I20260927 07:09:25.392513 10544 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:25.392556 10544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:25.392663 10544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:25.393710 10544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "6fe3e168416c4fcca881c4b5884b36f5"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "64710f4b5850fbd67480ccd90e067282"
server_key_iv: "33dbe4f7217807c4f59c650b3938811d"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.394021 10544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "6fe3e168416c4fcca881c4b5884b36f5"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "64710f4b5850fbd67480ccd90e067282"
server_key_iv: "33dbe4f7217807c4f59c650b3938811d"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.395328 10544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:25.396054 10559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.396209 10544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:25.396288 10544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "6fe3e168416c4fcca881c4b5884b36f5"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "64710f4b5850fbd67480ccd90e067282"
server_key_iv: "33dbe4f7217807c4f59c650b3938811d"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.396345 10544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:25.411494 10544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:25.413296 10544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:25.413424 10544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:25.413632 10544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:25.413915 10544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:25.413949 10544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.413977 10544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:25.413995 10544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.419324 10544 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:42185
I20260927 07:09:25.419389 10672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:42185 every 8 connection(s)
I20260927 07:09:25.419660 10544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:25.425977 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 10544
I20260927 07:09:25.426087 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:25.426273 4793 external_mini_cluster.cc:1428] Setting key 4e5b2561727ad1fc5eaae6f3242c58a8
I20260927 07:09:25.428272 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.420329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:58541 (local address 127.4.174.126:45187)
0927 07:09:25.420382 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.420384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.420579 (+ 195us) server_negotiation.cc:407] Connection header received
0927 07:09:25.421443 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.421444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.421454 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.421506 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.422050 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.422535 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.423472 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.423651 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.423698 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.425572 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:25.425574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:25.425574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:25.425580 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:25.426917 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.427337 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.427338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.427338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.427386 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.427612 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.427613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.427613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.427707 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:25.427750 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:25.428096 (+ 346us) server_negotiation.cc:299] Negotiation successful
0927 07:09:25.428175 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 07:09:25.428570 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.420301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45187 (local address 127.4.174.66:58541)
0927 07:09:25.420459 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:25.420500 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:25.421363 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:25.421567 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:25.421570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:25.421812 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:25.421991 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:25.421995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.422628 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.422629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:25.423410 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:25.423412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.423713 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.423714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:25.423818 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.424257 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:25.424267 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:25.425485 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:25.426983 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:25.426989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:25.426991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:25.427275 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:25.427439 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:25.427440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:25.427440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:25.427564 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:25.427755 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:25.427759 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:25.427904 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 07:09:25.428009 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 07:09:25.428156 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260927 07:09:25.428983 10673 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45187
I20260927 07:09:25.429105 10673 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:25.429309 10673 heartbeater.cc:511] Master 127.4.174.126:45187 requested a full tablet report, sending...
I20260927 07:09:25.429639 10348 ts_manager.cc:194] Registered new tserver with Master: 6fe3e168416c4fcca881c4b5884b36f5 (127.4.174.66:42185)
I20260927 07:09:25.430001 10348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:58541
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:25.449718 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45187
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:25.529443 10678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:25.529651 10678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:25.529685 10678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:25.530862 10678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:25.530908 10678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:25.530946 10678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:25.532405 10678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:25.532637 10678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:25.532850 10678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:25.537570 10684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:25.537670 10686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:25.537734 10678 server_base.cc:1034] running on GCE node
W20260927 07:09:25.537801 10683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:25.538004 10678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:25.538215 10678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:25.539366 10678 hybrid_clock.cc:648] HybridClock initialized: now 1790492965539347 us; error 30 us; skew 500 ppm
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.541769 10678 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:25.542187 10678 webserver.cc:460] Webserver started at http://127.4.174.67:39731/ using document root <none> and password file <none>
I20260927 07:09:25.542384 10678 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:25.542431 10678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:25.542572 10678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:25.543565 10678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "7b31556ee38d4485bf3d4f992446a366"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "2068aedea28b312cf028676f5aacd58c"
server_key_iv: "d14cb079dcd32e6172fdf642cf2d6cb5"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.543910 10678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "7b31556ee38d4485bf3d4f992446a366"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "2068aedea28b312cf028676f5aacd58c"
server_key_iv: "d14cb079dcd32e6172fdf642cf2d6cb5"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.545073 10678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:25.545667 10693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.545825 10678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:25.545902 10678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "7b31556ee38d4485bf3d4f992446a366"
format_stamp: "Formatted at 2026-09-27 07:09:25 on dist-test-slave-zw7p"
server_key: "2068aedea28b312cf028676f5aacd58c"
server_key_iv: "d14cb079dcd32e6172fdf642cf2d6cb5"
server_key_version: "encryptionkey@0"
I20260927 07:09:25.545960 10678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:25.565245 10678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:25.567219 10678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:25.567363 10678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:25.567591 10678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:25.567900 10678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:25.567936 10678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.567970 10678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:25.567993 10678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:25.573905 10678 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:40721
I20260927 07:09:25.573966 10806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:40721 every 8 connection(s)
I20260927 07:09:25.574285 10678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:25.583233 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.575005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:51791 (local address 127.4.174.126:45187)
0927 07:09:25.575070 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.575073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.575252 (+ 179us) server_negotiation.cc:407] Connection header received
0927 07:09:25.576173 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.576174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.576184 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.576236 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.576818 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.577303 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.578348 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.578538 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.578655 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.580725 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:25.580726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:25.580727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:25.580734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:25.581975 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.582356 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.582357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.582357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.582404 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.582638 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.582638 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.582638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.582740 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:25.582788 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:25.583054 (+ 266us) server_negotiation.cc:299] Negotiation successful
0927 07:09:25.583128 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 07:09:25.583312 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.574960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45187 (local address 127.4.174.67:51791)
0927 07:09:25.575129 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:25.575168 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:25.576075 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:25.576299 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:25.576302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:25.576552 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:25.576748 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:25.576753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.577393 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.577395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:25.578240 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:25.578243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.578658 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.578659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:25.578781 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.579233 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:25.579241 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:25.580630 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:25.582057 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:25.582063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:25.582065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:25.582295 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:25.582450 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:25.582451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:25.582451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:25.582541 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:25.582786 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:25.582790 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:25.582908 (+ 118us) client_negotiation.cc:769] Sending connection context
0927 07:09:25.582990 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 07:09:25.583109 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260927 07:09:25.584337 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 10678
I20260927 07:09:25.584188 10807 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45187
I20260927 07:09:25.584446 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:25.584463 10807 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:25.584635 4793 external_mini_cluster.cc:1428] Setting key 0a4284f488a11b06da024d457086ffa6
I20260927 07:09:25.584676 10807 heartbeater.cc:511] Master 127.4.174.126:45187 requested a full tablet report, sending...
I20260927 07:09:25.585201 10348 ts_manager.cc:194] Registered new tserver with Master: 7b31556ee38d4485bf3d4f992446a366 (127.4.174.67:40721)
I20260927 07:09:25.585741 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:09:25.585752 10348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:51791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/test-admin.keytab.
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.602707 4793 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260927 07:09:25.604310 10814 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:25.603539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.4.174.65:44127)
0927 07:09:25.603705 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.603709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.603726 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:09:25.603753 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.603755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.603771 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.603842 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.604221 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44766: BlockingRecv error: recv got EOF from 127.0.0.1:44766 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
W20260927 07:09:25.604362 4793 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5cc' of type FILE
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.605227 4793 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.606029 4793 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:25.613114 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.606547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.4.174.65:44127)
0927 07:09:25.606649 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.606650 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.606658 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:25.606695 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.606695 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.606706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.606750 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.607107 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.607836 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.608529 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.608862 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.608917 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.610927 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:25.610934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:25.610935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:25.610941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:25.612202 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.612498 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.612500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.612500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.612548 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.612699 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.612700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.612700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.612818 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:25.612914 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:25.612985 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 07:09:25.613040 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260927 07:09:25.614647 10814 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:25.614266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.4.174.65:44127)
0927 07:09:25.614300 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.614302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.614310 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:25.614348 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.614348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.614356 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.614395 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.614612 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44778: BlockingRecv error: recv got EOF from 127.0.0.1:44778 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 07:09:25.614737 4793 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5cc' of type FILE
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.615617 4793 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.616370 4793 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:25.622349 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.616821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44794 (local address 127.4.174.65:44127)
0927 07:09:25.616875 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.616876 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.616906 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:09:25.616968 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.616969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.616979 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.617021 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.617342 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.617809 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.618457 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.618686 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.618741 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.620409 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:25.620410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:25.620410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:25.620417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:25.621445 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.621724 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.621725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.621725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.621771 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.621924 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.621925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.621925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.622010 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:25.622087 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:25.622191 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 07:09:25.622261 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260927 07:09:25.623545 10814 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:25.623170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.4.174.65:44127)
0927 07:09:25.623203 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.623205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.623237 (+ 32us) server_negotiation.cc:407] Connection header received
0927 07:09:25.623281 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.623281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.623288 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.623325 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.623504 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44800: BlockingRecv error: recv got EOF from 127.0.0.1:44800 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 07:09:25.623654 4793 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492940996973-4793-0/krb5kdc/krb5cc' of type FILE
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.624444 4793 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:25.625149 4793 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.65@KRBTEST.COM
I20260927 07:09:25.630995 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:25.625562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44810 (local address 127.4.174.65:44127)
0927 07:09:25.625595 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:25.625597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:25.625604 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:25.625672 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:25.625673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:25.625681 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:25.625723 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:25.626035 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.626529 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.627229 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:25.627401 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:25.627451 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:25.629093 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:25.629094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:25.629095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:25.629101 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:25.630075 (+ 974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.630354 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.630355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.630355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.630399 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:25.630572 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:25.630572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:25.630572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:25.630659 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:25.630736 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:25.630843 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 07:09:25.630910 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:09:25.631525 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 10410
I20260927 07:09:25.635967 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 10544
I20260927 07:09:25.639730 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 10678
I20260927 07:09:25.643158 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 10316
2026-09-27T07:09:25Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1748 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10827](info): set up 2 sockets
Sep 27 07:09:25 dist-test-slave-zw7p krb5kdc[10827](info): commencing operation
krb5kdc: starting...
W20260927 07:09:26.606408 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:26 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.126.keytab.
Entry for principal oryx/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.126.keytab.
I20260927 07:09:26.716917 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:33097
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.126.keytab
--principal=oryx/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:33097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:26.794658 10840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:26.794857 10840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:26.794878 10840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:26.795989 10840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:26.796020 10840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:26.796032 10840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:26.796043 10840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:26.796054 10840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:26.797454 10840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:33097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.126.keytab
--principal=oryx/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:33097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:26.797649 10840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:26.797819 10840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:26.802282 10848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:26.802386 10840 server_base.cc:1034] running on GCE node
W20260927 07:09:26.802284 10846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:26.802289 10845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:26.802784 10840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:26.803040 10840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:26.804173 10840 hybrid_clock.cc:648] HybridClock initialized: now 1790492966804161 us; error 28 us; skew 500 ppm
Sep 27 07:09:26 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:26.806589 10840 init.cc:377] Logged in from keytab as oryx/127.4.174.126@KRBTEST.COM (short username oryx)
I20260927 07:09:26.807070 10840 webserver.cc:460] Webserver started at http://127.4.174.126:45763/ using document root <none> and password file <none>
I20260927 07:09:26.807323 10840 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:26.807386 10840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:26.807524 10840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:26.808871 10840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "bfd23aea91a3461894fdd3149d49b4d7"
format_stamp: "Formatted at 2026-09-27 07:09:26 on dist-test-slave-zw7p"
server_key: "79828f2f77af6226933d2b9728061ab5"
server_key_iv: "0a72bd3b8f08e43aff65a33b08f260f3"
server_key_version: "encryptionkey@0"
I20260927 07:09:26.809239 10840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "bfd23aea91a3461894fdd3149d49b4d7"
format_stamp: "Formatted at 2026-09-27 07:09:26 on dist-test-slave-zw7p"
server_key: "79828f2f77af6226933d2b9728061ab5"
server_key_iv: "0a72bd3b8f08e43aff65a33b08f260f3"
server_key_version: "encryptionkey@0"
I20260927 07:09:26.810467 10840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:26.811194 10855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:26.811337 10840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:26.811417 10840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "bfd23aea91a3461894fdd3149d49b4d7"
format_stamp: "Formatted at 2026-09-27 07:09:26 on dist-test-slave-zw7p"
server_key: "79828f2f77af6226933d2b9728061ab5"
server_key_iv: "0a72bd3b8f08e43aff65a33b08f260f3"
server_key_version: "encryptionkey@0"
I20260927 07:09:26.811482 10840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:26.821027 10840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:26.823125 10840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:26.823246 10840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:26.827070 10840 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:33097
I20260927 07:09:26.827111 10907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:33097 every 8 connection(s)
I20260927 07:09:26.827457 10840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:26.828435 10908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:26.830667 10908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: Bootstrap starting.
I20260927 07:09:26.831207 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 10840
I20260927 07:09:26.831235 10908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:26.831298 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:26.831468 4793 external_mini_cluster.cc:1428] Setting key 53a8a5055d85480cb91701bd022c309f
I20260927 07:09:26.831488 10908 log.cc:824] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:26.832190 10908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: No bootstrap required, opened a new log
I20260927 07:09:26.834671 10908 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } }
I20260927 07:09:26.834836 10908 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:26.834868 10908 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd23aea91a3461894fdd3149d49b4d7, State: Initialized, Role: FOLLOWER
I20260927 07:09:26.834988 10908 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } }
I20260927 07:09:26.835072 10908 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:26.835094 10908 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:26.835141 10908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:26.835659 10908 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } }
I20260927 07:09:26.835759 10908 leader_election.cc:302] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfd23aea91a3461894fdd3149d49b4d7; no voters:
I20260927 07:09:26.835908 10908 leader_election.cc:288] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:26.835955 10913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:26.836114 10913 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 1 LEADER]: Becoming Leader. State: Replica: bfd23aea91a3461894fdd3149d49b4d7, State: Running, Role: LEADER
I20260927 07:09:26.836145 10908 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:26.836211 10913 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } }
I20260927 07:09:26.836784 10914 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } }
I20260927 07:09:26.836802 10915 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd23aea91a3461894fdd3149d49b4d7. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } }
I20260927 07:09:26.836869 10914 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:26.836884 10915 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:26.837096 10922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:26.837564 10922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 07:09:26 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:26.839362 10922 catalog_manager.cc:1347] Generated new cluster ID: 841d0267662c4122bf6520a898268acc
I20260927 07:09:26.839459 10922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:26.842741 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:26.831933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53606 (local address 127.4.174.126:33097)
0927 07:09:26.832101 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:26.832110 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:26.832181 (+ 71us) server_negotiation.cc:407] Connection header received
0927 07:09:26.832677 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:26.832687 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:26.832752 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:26.832912 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:26.833456 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:26.834860 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:26.835738 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:26.836335 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:26.836481 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:26.839502 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:26.839506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:26.839506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:26.839517 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:26.841169 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:26.841503 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:26.841507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:26.841507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:26.841600 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:26.841781 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:26.841782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:26.841783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:26.842077 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:26.842242 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:26.842352 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 07:09:26.842484 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260927 07:09:26.859978 10922 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:26.860502 10922 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:26.862483 10922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: Generated new TSK 0
I20260927 07:09:26.862676 10922 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab.
Entry for principal oryx/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab.
I20260927 07:09:26.889557 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33097
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab
--principal=oryx/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:26.971911 10934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:26.972119 10934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:26.972151 10934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:26.973297 10934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:26.973337 10934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:26.973371 10934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:26.974879 10934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab
--principal=oryx/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:26.975112 10934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:26.975327 10934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:26.980075 10940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:26.980079 10939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:26.980036 10942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:26.980490 10934 server_base.cc:1034] running on GCE node
I20260927 07:09:26.980676 10934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:26.980899 10934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:26.982033 10934 hybrid_clock.cc:648] HybridClock initialized: now 1790492966982020 us; error 30 us; skew 500 ppm
Sep 27 07:09:26 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:26.984534 10934 init.cc:377] Logged in from keytab as oryx/127.4.174.65@KRBTEST.COM (short username oryx)
I20260927 07:09:26.984922 10934 webserver.cc:460] Webserver started at http://127.4.174.65:46331/ using document root <none> and password file <none>
I20260927 07:09:26.985142 10934 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:26.985185 10934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:26.985280 10934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:26.986266 10934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea"
format_stamp: "Formatted at 2026-09-27 07:09:26 on dist-test-slave-zw7p"
server_key: "2f8021f43aac8f51eb0a9f6651a1cc09"
server_key_iv: "098a52ad838ad9cc1be0bc0c7872256e"
server_key_version: "encryptionkey@0"
I20260927 07:09:26.986583 10934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea"
format_stamp: "Formatted at 2026-09-27 07:09:26 on dist-test-slave-zw7p"
server_key: "2f8021f43aac8f51eb0a9f6651a1cc09"
server_key_iv: "098a52ad838ad9cc1be0bc0c7872256e"
server_key_version: "encryptionkey@0"
I20260927 07:09:26.987787 10934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:26.988446 10949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:26.988601 10934 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:26.988679 10934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea"
format_stamp: "Formatted at 2026-09-27 07:09:26 on dist-test-slave-zw7p"
server_key: "2f8021f43aac8f51eb0a9f6651a1cc09"
server_key_iv: "098a52ad838ad9cc1be0bc0c7872256e"
server_key_version: "encryptionkey@0"
I20260927 07:09:26.988732 10934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.017109 10934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:27.019254 10934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:27.019384 10934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:27.019583 10934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:27.019857 10934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:27.019888 10934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.019910 10934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:27.019927 10934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.025203 10934 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:44313
I20260927 07:09:27.025272 11062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:44313 every 8 connection(s)
I20260927 07:09:27.025571 10934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:27.034719 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.026287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:41421 (local address 127.4.174.126:33097)
0927 07:09:27.026370 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.026372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.026575 (+ 203us) server_negotiation.cc:407] Connection header received
0927 07:09:27.027086 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.027087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.027099 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.027158 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.028135 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.028638 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.029611 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.029847 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.029911 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.031978 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.031979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.031980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.031987 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.033405 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.033719 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.033720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.033720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.033772 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.033986 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.033987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.033987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.034155 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.034227 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.034490 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.034620 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:09:27.034734 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 10934
I20260927 07:09:27.034736 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.026265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:41421)
0927 07:09:27.026435 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.026472 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.026982 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.027215 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.027218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.027772 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.027967 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.027972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.028713 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.028714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.029523 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.029525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.029918 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.029919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.030019 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.030477 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.030486 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.031890 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.033473 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.033477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.033478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.033668 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.033818 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.033818 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.033819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.033906 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.034215 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.034218 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.034329 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.034408 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.034525 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260927 07:09:27.034817 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:27.035003 4793 external_mini_cluster.cc:1428] Setting key 05aa0bde1086a57bc120b54c7b8be623
I20260927 07:09:27.035569 11063 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33097
I20260927 07:09:27.035681 11063 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:27.035884 11063 heartbeater.cc:511] Master 127.4.174.126:33097 requested a full tablet report, sending...
I20260927 07:09:27.036459 10872 ts_manager.cc:194] Registered new tserver with Master: 6e2ac51ab50e41cd8d637a2cd42d19ea (127.4.174.65:44313)
I20260927 07:09:27.037201 10872 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:41421
WARNING: no policy specified for oryx/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab.
Entry for principal oryx/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab.
I20260927 07:09:27.059409 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33097
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab
--principal=oryx/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:27.138443 11068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:27.138693 11068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:27.138727 11068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:27.139919 11068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:27.139959 11068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:27.139994 11068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:27.141453 11068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab
--principal=oryx/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:27.141698 11068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:27.141917 11068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:27.146673 11074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.146725 11073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.146708 11076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.146914 11068 server_base.cc:1034] running on GCE node
I20260927 07:09:27.147127 11068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:27.147315 11068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:27.148458 11068 hybrid_clock.cc:648] HybridClock initialized: now 1790492967148440 us; error 27 us; skew 500 ppm
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:27.150803 11068 init.cc:377] Logged in from keytab as oryx/127.4.174.66@KRBTEST.COM (short username oryx)
I20260927 07:09:27.151194 11068 webserver.cc:460] Webserver started at http://127.4.174.66:36053/ using document root <none> and password file <none>
I20260927 07:09:27.151362 11068 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.151399 11068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.151486 11068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:27.152472 11068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "f8d46f0507454efe97b2840b0a6016ca"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "48ab42988717542295362d896bc8bc2b"
server_key_iv: "9fee7fec9bd323b30be80d9a5989faec"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.152763 11068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "f8d46f0507454efe97b2840b0a6016ca"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "48ab42988717542295362d896bc8bc2b"
server_key_iv: "9fee7fec9bd323b30be80d9a5989faec"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.153961 11068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:27.154627 11083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.154769 11068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:27.154845 11068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "f8d46f0507454efe97b2840b0a6016ca"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "48ab42988717542295362d896bc8bc2b"
server_key_iv: "9fee7fec9bd323b30be80d9a5989faec"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.154898 11068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.165385 11068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:27.167387 11068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:27.167490 11068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:27.167665 11068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:27.167917 11068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:27.167945 11068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.167965 11068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:27.167981 11068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.173204 11068 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:43757
I20260927 07:09:27.173266 11196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:43757 every 8 connection(s)
I20260927 07:09:27.173565 11068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:27.173992 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 11068
I20260927 07:09:27.174081 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:27.174278 4793 external_mini_cluster.cc:1428] Setting key 628168b2ad3d7e08bf1c07a341e29601
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.4.174.67@KRBTEST.COM; defaulting to no policy
I20260927 07:09:27.182511 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.174292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:43543 (local address 127.4.174.126:33097)
0927 07:09:27.174353 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.174355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.174539 (+ 184us) server_negotiation.cc:407] Connection header received
0927 07:09:27.175189 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.175190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.175201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.175251 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.176158 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.176700 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.177656 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.177851 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.177904 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.179893 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.179895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.179895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.179902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.181205 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.181628 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.181629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.181630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.181686 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.181898 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.181899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.181899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.182012 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.182068 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.182370 (+ 302us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.182441 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 07:09:27.182698 11199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.174264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:43543)
0927 07:09:27.174415 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.174455 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.175092 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.175320 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.175324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.175870 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.176066 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.176071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.176783 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.176783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.177575 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.177577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.177918 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.177919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.178047 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.178521 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.178529 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.179811 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.181289 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.181297 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.181299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.181555 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.181739 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.181740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.181741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.181844 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.182065 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.182071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.182233 (+ 162us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.182342 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.182470 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260927 07:09:27.183128 11197 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33097
I20260927 07:09:27.183214 11197 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:27.183358 11197 heartbeater.cc:511] Master 127.4.174.126:33097 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.67@KRBTEST.COM" created.
I20260927 07:09:27.183670 10872 ts_manager.cc:194] Registered new tserver with Master: f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757)
I20260927 07:09:27.184026 10872 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:43543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab.
Entry for principal oryx/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab.
I20260927 07:09:27.198007 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33097
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab
--principal=oryx/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:27.276437 11202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:27.276633 11202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:27.276654 11202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:27.277755 11202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:27.277796 11202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:27.277823 11202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:27.279256 11202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab
--principal=oryx/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:27.279474 11202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:27.279672 11202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:27.284593 11207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.284631 11208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.284742 11202 server_base.cc:1034] running on GCE node
W20260927 07:09:27.284581 11210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.284991 11202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:27.285234 11202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:27.286396 11202 hybrid_clock.cc:648] HybridClock initialized: now 1790492967286387 us; error 33 us; skew 500 ppm
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:27.288945 11202 init.cc:377] Logged in from keytab as oryx/127.4.174.67@KRBTEST.COM (short username oryx)
I20260927 07:09:27.289387 11202 webserver.cc:460] Webserver started at http://127.4.174.67:39381/ using document root <none> and password file <none>
I20260927 07:09:27.289624 11202 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.289680 11202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.289812 11202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:27.290935 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "d64dde1fc3b94d059545eb443a4e85c9"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "499f86aa09fc1ffd63f12040e7f5d989"
server_key_iv: "d4d6c337858d4708b557f2cf8b72e752"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.291311 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "d64dde1fc3b94d059545eb443a4e85c9"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "499f86aa09fc1ffd63f12040e7f5d989"
server_key_iv: "d4d6c337858d4708b557f2cf8b72e752"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.292667 11202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:27.293431 11217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.293630 11202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:27.293720 11202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "d64dde1fc3b94d059545eb443a4e85c9"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "499f86aa09fc1ffd63f12040e7f5d989"
server_key_iv: "d4d6c337858d4708b557f2cf8b72e752"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.293783 11202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.301785 11202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:27.303481 11202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:27.303614 11202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:27.303803 11202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:27.304078 11202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:27.304112 11202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.304140 11202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:27.304163 11202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.309370 11202 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:41547
I20260927 07:09:27.309432 11330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:41547 every 8 connection(s)
I20260927 07:09:27.309700 11202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:27.312320 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 11202
I20260927 07:09:27.312404 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:27.312556 4793 external_mini_cluster.cc:1428] Setting key 63b5ac8023d635d749db0a6acddff3a3
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:27.318252 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.310313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:45833 (local address 127.4.174.126:33097)
0927 07:09:27.310355 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.310357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.310575 (+ 218us) server_negotiation.cc:407] Connection header received
0927 07:09:27.311101 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.311103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.311113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.311175 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.312092 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.312697 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.313855 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.314041 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.314115 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.316081 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.316082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.316082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.316088 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.317185 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.317501 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.317502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.317502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.317547 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.317723 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.317724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.317724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.317817 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.317860 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.318108 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.318183 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:09:27.318315 11333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.310285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:45833)
0927 07:09:27.310427 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.310462 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.310966 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.311255 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.311258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.311805 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.311991 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.311996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.312767 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.312769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.313784 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.313786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.314121 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.314122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.314236 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.314732 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.314740 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.315996 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.317247 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.317252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.317254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.317449 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.317590 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.317591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.317591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.317681 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.317858 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.317860 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.317965 (+ 105us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.318041 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.318159 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260927 07:09:27.318758 11331 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33097
I20260927 07:09:27.318851 11331 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:27.319036 11331 heartbeater.cc:511] Master 127.4.174.126:33097 requested a full tablet report, sending...
I20260927 07:09:27.319386 10872 ts_manager.cc:194] Registered new tserver with Master: d64dde1fc3b94d059545eb443a4e85c9 (127.4.174.67:41547)
I20260927 07:09:27.319754 10872 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:45833
I20260927 07:09:27.324028 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.174.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.125.keytab.
Entry for principal oryx/127.4.174.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.125.keytab.
I20260927 07:09:27.346843 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.125:37471
--webserver_interface=127.4.174.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.125.keytab
--principal=oryx/127.4.174.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:33097,127.4.174.125:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:27.425488 11336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:27.425693 11336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:27.425714 11336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:27.426837 11336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:27.426875 11336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:27.426887 11336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:27.426899 11336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:27.426914 11336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:27.428323 11336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:33097,127.4.174.125:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.125.keytab
--principal=oryx/127.4.174.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.125:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.174.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
W20260927 07:09:27.428419 11336 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.174.126:33097,127.4.174.125:37471'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260927 07:09:27.428532 11336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:27.428704 11336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:27.433218 11341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.433265 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.433269 11342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.433465 11336 server_base.cc:1034] running on GCE node
I20260927 07:09:27.433702 11336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:27.434000 11336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:27.435165 11336 hybrid_clock.cc:648] HybridClock initialized: now 1790492967435146 us; error 32 us; skew 500 ppm
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:27.437637 11336 init.cc:377] Logged in from keytab as oryx/127.4.174.125@KRBTEST.COM (short username oryx)
I20260927 07:09:27.438072 11336 webserver.cc:460] Webserver started at http://127.4.174.125:45995/ using document root <none> and password file <none>
I20260927 07:09:27.438266 11336 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.438313 11336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.438432 11336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:27.439544 11336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data/instance:
uuid: "65020500fc70441b97bf3547b2ed780b"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "46c6c35457680257dad12f1198a993cf"
server_key_iv: "f1410d81de0dc6dfd1b4d5368fe5d01f"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.439901 11336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal/instance:
uuid: "65020500fc70441b97bf3547b2ed780b"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "46c6c35457680257dad12f1198a993cf"
server_key_iv: "f1410d81de0dc6dfd1b4d5368fe5d01f"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.441143 11336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:27.441874 11351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.442035 11336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:27.442127 11336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
uuid: "65020500fc70441b97bf3547b2ed780b"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "46c6c35457680257dad12f1198a993cf"
server_key_iv: "f1410d81de0dc6dfd1b4d5368fe5d01f"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.442203 11336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.455166 11336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:27.457010 11336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:27.457140 11336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:27.460994 11336 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.125:37471
I20260927 07:09:27.461045 11403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.125:37471 every 8 connection(s)
I20260927 07:09:27.461373 11336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
I20260927 07:09:27.462347 11404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:27.463977 11404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.125@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:27.471302 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 11336
I20260927 07:09:27.472409 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.464712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53612 (local address 127.4.174.126:33097)
0927 07:09:27.464787 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.464790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.464967 (+ 177us) server_negotiation.cc:407] Connection header received
0927 07:09:27.465443 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.465445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.465457 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.465508 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.466098 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.466651 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.467605 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.467806 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.467861 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.469861 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.469862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.469863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.469870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.471009 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.471353 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.471354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.471355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.471416 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.471698 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.471699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.471699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.471901 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.471965 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.472241 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.472324 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 07:09:27.472564 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.464662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:53612)
0927 07:09:27.464853 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.464889 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.465350 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.465574 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.465578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.465827 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.466011 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.466015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.466736 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.466738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.467523 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.467525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.467872 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.467873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.467989 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.468446 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.468454 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.469776 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.471077 (+ 1301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.471084 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.471086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.471288 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.471482 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.471483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.471483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.471630 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.471965 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.471972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.472093 (+ 121us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.472205 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.472355 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260927 07:09:27.473212 11404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.125@KRBTEST.COM for oryx/127.4.174.125@KRBTEST.COM
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.174.125@KRBTEST.COM
I20260927 07:09:27.479426 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.473434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.0.0.1:51786)
0927 07:09:27.473484 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.473493 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.473517 (+ 24us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.473721 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.473721 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.473840 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.473883 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.473885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.474461 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.474462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.475008 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.475009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.475315 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.475316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.475387 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.475683 (+ 296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.475687 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.476764 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.478347 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.478348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.478349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.478579 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.478770 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.478770 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.478771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.478828 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.479110 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.479111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.479151 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.479226 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.479331 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 07:09:27.479522 11409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.473503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.4.174.125:37471)
0927 07:09:27.473636 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.473638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.473643 (+ 5us) server_negotiation.cc:407] Connection header received
0927 07:09:27.473655 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.473655 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.473662 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.473714 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.473939 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.474399 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.475074 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.475254 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.475299 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.476854 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.476860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.476860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.476865 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.478273 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.478652 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.478655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.478655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.478715 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.478889 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.478890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.478890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.479041 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.479100 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.479250 (+ 150us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.479412 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260927 07:09:27.480561 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.471772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51770 (local address 127.4.174.125:37471)
0927 07:09:27.471923 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.471928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.471973 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:09:27.472061 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.472065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.472077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.472144 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.472929 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.473806 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.474545 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.474906 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.474964 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.477466 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.477468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.477468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.477474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.479523 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.479871 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.479872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.479872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.479918 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.480104 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.480104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.480105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.480197 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.480287 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.480412 (+ 125us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.480466 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260927 07:09:27.481173 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Bootstrap starting.
I20260927 07:09:27.481709 11404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:27.481971 11404 log.cc:824] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:27.482611 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: No bootstrap required, opened a new log
I20260927 07:09:27.483911 11404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } }
I20260927 07:09:27.484028 11404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:27.484058 11404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65020500fc70441b97bf3547b2ed780b, State: Initialized, Role: FOLLOWER
I20260927 07:09:27.484181 11404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } }
I20260927 07:09:27.484439 11404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:27.484407 11411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } } }
I20260927 07:09:27.484483 11411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: This master's current role is: FOLLOWER
W20260927 07:09:27.485822 11422 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:09:27.485878 11422 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:09:27.487340 11336 master_runner.cc:428] Detected that this master 65020500fc70441b97bf3547b2ed780b is joining an existing cluster
I20260927 07:09:27.487385 11336 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.174.125:37471
I20260927 07:09:27.494174 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.488648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51792 (local address 127.4.174.125:37471)
0927 07:09:27.488683 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.488684 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.488773 (+ 89us) server_negotiation.cc:407] Connection header received
0927 07:09:27.488842 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.488843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.488850 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.488889 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.489214 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.489712 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.490419 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.490703 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.490757 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.491569 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.491570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.491570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.491576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.493161 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.493478 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.493479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.493479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.493530 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.493695 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.493696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.493696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.493792 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.493843 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.493957 (+ 114us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.494089 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":19}
I20260927 07:09:27.494186 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.488591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.0.0.1:51792)
0927 07:09:27.488704 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.488714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.488773 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.488891 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.488892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.489078 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.489151 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.489153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.489769 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.489770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.490352 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.490354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.490762 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.490763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.490820 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.491211 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.491215 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.491491 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.493237 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.493238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.493238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.493418 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.493582 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.493582 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.493583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.493648 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.493840 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.493840 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.493876 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.493934 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.494032 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260927 07:09:27.495153 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.488439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53618 (local address 127.4.174.126:33097)
0927 07:09:27.488479 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.488481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.488812 (+ 331us) server_negotiation.cc:407] Connection header received
0927 07:09:27.488836 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.488837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.488848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.488894 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.489287 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.490036 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.491138 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.491419 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.491459 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.492428 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.492430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.492430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.492437 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.493958 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.494350 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.494351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.494351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.494411 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.494666 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.494667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.494668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.494805 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.494845 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.495028 (+ 183us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.495094 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:09:27.495186 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.488356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:53618)
0927 07:09:27.488704 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.488714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.488773 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.488940 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.488941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.489136 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.489196 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.489198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.490128 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.490129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.491044 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.491046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.491531 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.491532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.491596 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.491944 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.491949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.492347 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.494046 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.494047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.494047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.494266 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.494478 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.494478 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.494479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.494543 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.494890 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.494891 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.494930 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.494990 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.495084 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":108,"threads_started":1}
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.126@KRBTEST.COM for oryx/127.4.174.125@KRBTEST.COM
I20260927 07:09:27.503255 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.497103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.4.174.125:37471)
0927 07:09:27.497143 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.497145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.497299 (+ 154us) server_negotiation.cc:407] Connection header received
0927 07:09:27.497405 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.497405 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.497429 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.497470 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.497861 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.498307 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.499077 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.499270 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.499322 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.501087 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.501088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.501088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.501095 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.502241 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.502576 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.502577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.502578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.502628 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.502783 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.502784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.502784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.502882 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.502941 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.503092 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.503163 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:09:27.503270 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.497087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.0.0.1:51804)
0927 07:09:27.497209 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.497252 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.497318 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.497472 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.497475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.497745 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.497792 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.497794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.498369 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.498371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.499005 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.499007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.499326 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.499326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.499419 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.499797 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.499802 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.501009 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.502309 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.502316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.502318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.502496 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.502679 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.502680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.502680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.502736 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.502932 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.502940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.502990 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.503062 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.503167 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":79,"threads_started":1}
I20260927 07:09:27.503911 10872 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.174.125:37471: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "65020500fc70441b97bf3547b2ed780b"
member_type: NON_VOTER
last_known_addr {
host: "127.4.174.125"
port: 37471
}
attrs {
promote: true
}
}
dest_uuid: "bfd23aea91a3461894fdd3149d49b4d7"
cas_config_opid_index: -1
I20260927 07:09:27.504217 10872 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } }
I20260927 07:09:27.504647 11432 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 65020500fc70441b97bf3547b2ed780b (127.4.174.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } }
I20260927 07:09:27.504941 11378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:27.505470 11432 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.174.125:37471
I20260927 07:09:27.505558 11432 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } }
I20260927 07:09:27.505628 11432 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:27.505772 11336 master.cc:537] Master@127.4.174.125:37471 shutting down...
I20260927 07:09:27.505865 11378 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Refusing update from remote peer bfd23aea91a3461894fdd3149d49b4d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260927 07:09:27.506181 11431 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 07:09:27.506868 10858 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260927 07:09:27.507256 10858 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 -> Peer 65020500fc70441b97bf3547b2ed780b (127.4.174.125:37471): Couldn't send request to peer 65020500fc70441b97bf3547b2ed780b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 07:09:27.508515 11336 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:27.508597 11336 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:27.508647 11336 tablet_replica.cc:330] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: stopping tablet replica
I20260927 07:09:27.521461 11336 master.cc:552] Master@127.4.174.125:37471 shutdown complete.
I20260927 07:09:27.521522 11336 master_runner.cc:305] Clearing existing system tablet
I20260927 07:09:27.521661 11336 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.521711 11336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.522446 11336 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260927 07:09:27.523005 11436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.523138 11336 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.523198 11336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.523221 11336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
uuid: "65020500fc70441b97bf3547b2ed780b"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "46c6c35457680257dad12f1198a993cf"
server_key_iv: "f1410d81de0dc6dfd1b4d5368fe5d01f"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.523912 11336 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:27.525302 11336 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 07:09:27.525375 11336 log.cc:1196] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260927 07:09:27.525552 11336 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Deleting consensus metadata
I20260927 07:09:27.525652 11336 master_runner.cc:315] Copying system tablet from 127.4.174.126:33097
I20260927 07:09:27.526217 11336 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: tablet copy: Beginning tablet copy session from remote peer at address 127.4.174.126:33097
I20260927 07:09:27.532521 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.526621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.4.174.126:33097)
0927 07:09:27.526663 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.526665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.526752 (+ 87us) server_negotiation.cc:407] Connection header received
0927 07:09:27.526829 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.526829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.526837 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.526883 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.527231 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.527742 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.528771 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.528967 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.529020 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.530066 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.530067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.530068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.530074 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.531524 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.531875 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.531876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.531876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.531927 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.532093 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.532093 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.532094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.532189 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.532232 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.532368 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.532420 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:09:27.532557 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.526495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:53622)
0927 07:09:27.526685 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.526696 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.526769 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.526890 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.526891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.527083 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.527158 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.527160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.527811 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.527812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.528685 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.528687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.529032 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.529033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.529099 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.529568 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.529572 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.529958 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.531590 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.531591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.531592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.531809 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.531979 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.531979 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.531980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.532040 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.532237 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.532238 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.532285 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.532351 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.532446 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":78,"threads_started":1}
I20260927 07:09:27.532933 10892 tablet_copy_service.cc:140] P bfd23aea91a3461894fdd3149d49b4d7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 65020500fc70441b97bf3547b2ed780b ({username='oryx', principal='oryx/127.4.174.125@KRBTEST.COM'} at 127.0.0.1:53622)
I20260927 07:09:27.533021 10892 tablet_copy_service.cc:161] P bfd23aea91a3461894fdd3149d49b4d7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 65020500fc70441b97bf3547b2ed780b at {username='oryx', principal='oryx/127.4.174.125@KRBTEST.COM'} at 127.0.0.1:53622: session id = 65020500fc70441b97bf3547b2ed780b-00000000000000000000000000000000
I20260927 07:09:27.533615 10892 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 07:09:27.534150 11336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 07:09:27.535080 11336 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: tablet copy: Starting download of 0 data blocks...
I20260927 07:09:27.535199 11336 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: tablet copy: Starting download of 1 WAL segments...
I20260927 07:09:27.535954 11336 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 07:09:27.536660 10892 tablet_copy_service.cc:342] P bfd23aea91a3461894fdd3149d49b4d7: Request end of tablet copy session 65020500fc70441b97bf3547b2ed780b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.174.125@KRBTEST.COM'} at 127.0.0.1:53622
I20260927 07:09:27.536712 10892 tablet_copy_service.cc:433] P bfd23aea91a3461894fdd3149d49b4d7: ending tablet copy session 65020500fc70441b97bf3547b2ed780b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 65020500fc70441b97bf3547b2ed780b
I20260927 07:09:27.539199 11336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:27.540264 11443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.540412 11444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.543396 11446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.543558 11336 server_base.cc:1034] running on GCE node
I20260927 07:09:27.543665 11336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:27.543797 11336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:27.544927 11336 hybrid_clock.cc:648] HybridClock initialized: now 1790492967544918 us; error 26 us; skew 500 ppm
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:27.546177 11336 init.cc:377] Logged in from keytab as oryx/127.4.174.125@KRBTEST.COM (short username oryx)
I20260927 07:09:27.546407 11336 webserver.cc:460] Webserver started at http://127.4.174.125:35001/ using document root <none> and password file <none>
I20260927 07:09:27.546491 11336 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.546514 11336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.547102 11336 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.547576 11453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.547685 11336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.547729 11336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
uuid: "65020500fc70441b97bf3547b2ed780b"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "46c6c35457680257dad12f1198a993cf"
server_key_iv: "f1410d81de0dc6dfd1b4d5368fe5d01f"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.547778 11336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.573398 11336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:27.574913 11336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:27.577893 11336 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.125:37471
I20260927 07:09:27.578258 11505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.125:37471 every 8 connection(s)
I20260927 07:09:27.578285 11336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
I20260927 07:09:27.579247 11506 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:09:27.579795 11506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Bootstrap starting.
I20260927 07:09:27.582911 11506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:09:27.583114 11506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: Bootstrap complete.
I20260927 07:09:27.583307 11506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } }
I20260927 07:09:27.583390 11506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 65020500fc70441b97bf3547b2ed780b, State: Initialized, Role: LEARNER
I20260927 07:09:27.583422 11506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } }
I20260927 07:09:27.583585 11506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:27.583590 11510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } }
I20260927 07:09:27.583837 11510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: This master's current role is: LEARNER
I20260927 07:09:27.585002 11521 catalog_manager.cc:1259] Loaded cluster ID: 841d0267662c4122bf6520a898268acc
I20260927 07:09:27.585390 11521 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: loading cluster ID for follower catalog manager: success
I20260927 07:09:27.586833 11521 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: acquiring CA information for follower catalog manager: success
I20260927 07:09:27.587149 11521 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:09:27.588706 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.581795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.4.174.125:37471)
0927 07:09:27.581922 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.581924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.581933 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:27.581952 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.581953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.581962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.582027 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.582540 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.583351 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.584335 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.584609 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.584947 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.586229 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.586230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.586231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.586237 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.587702 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.588000 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.588001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.588001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.588067 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.588227 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.588228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.588229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.588330 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.588426 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.588512 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.588598 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.4.174.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.124.keytab.
Entry for principal oryx/127.4.174.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.124.keytab.
I20260927 07:09:27.615199 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.124:45073
--webserver_interface=127.4.174.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.124.keytab
--principal=oryx/127.4.174.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:33097,127.4.174.125:37471,127.4.174.124:45073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:27.693871 11524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:27.694061 11524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:27.694082 11524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:27.695212 11524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:27.695251 11524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:27.695264 11524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:27.695287 11524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:27.695309 11524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:27.696834 11524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:33097,127.4.174.125:37471,127.4.174.124:45073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.124.keytab
--principal=oryx/127.4.174.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.124:45073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.174.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:27.697052 11524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:27.697283 11524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:27.701748 11532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.701749 11529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.701813 11530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.702028 11524 server_base.cc:1034] running on GCE node
I20260927 07:09:27.702244 11524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:27.702526 11524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:27.703725 11524 hybrid_clock.cc:648] HybridClock initialized: now 1790492967703710 us; error 29 us; skew 500 ppm
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:27.706095 11524 init.cc:377] Logged in from keytab as oryx/127.4.174.124@KRBTEST.COM (short username oryx)
I20260927 07:09:27.706509 11524 webserver.cc:460] Webserver started at http://127.4.174.124:42461/ using document root <none> and password file <none>
I20260927 07:09:27.706735 11524 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.706784 11524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.706904 11524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:27.707870 11524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data/instance:
uuid: "f8260de92b0b4045a00d10ee509503c5"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "a05328aaea8b2cf9e5b136b426933de5"
server_key_iv: "1ba0ab993e860a458f1a71d2754aa6c4"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.708195 11524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal/instance:
uuid: "f8260de92b0b4045a00d10ee509503c5"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "a05328aaea8b2cf9e5b136b426933de5"
server_key_iv: "1ba0ab993e860a458f1a71d2754aa6c4"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.709391 11524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:27.709982 11539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.710156 11524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:27.710235 11524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
uuid: "f8260de92b0b4045a00d10ee509503c5"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "a05328aaea8b2cf9e5b136b426933de5"
server_key_iv: "1ba0ab993e860a458f1a71d2754aa6c4"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.710294 11524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.723987 11524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:27.726058 11524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:27.726195 11524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:27.729872 11524 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.124:45073
I20260927 07:09:27.729915 11591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.124:45073 every 8 connection(s)
I20260927 07:09:27.730245 11524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
I20260927 07:09:27.731228 11592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:27.732724 11592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.124@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:27.739486 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 11524
I20260927 07:09:27.741711 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.733482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53626 (local address 127.4.174.126:33097)
0927 07:09:27.733542 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.733544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.733647 (+ 103us) server_negotiation.cc:407] Connection header received
0927 07:09:27.734134 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.734135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.734147 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.734199 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.734853 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.735357 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.736400 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.736625 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.736679 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.738802 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.738804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.738805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.738813 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.740260 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.740734 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.740735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.740735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.740791 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.741083 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.741084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.741084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.741215 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.741280 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.741522 (+ 242us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.741600 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:27.741794 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.733346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:53626)
0927 07:09:27.733513 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.733560 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.734029 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.734270 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.734273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.734517 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.734769 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.734773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.735445 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.735446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.736324 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.736327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.736709 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.736710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.736839 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.737321 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.737332 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.738712 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.740332 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.740336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.740338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.740642 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.740844 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.740845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.740845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.740979 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.741275 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.741279 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.741406 (+ 127us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.741487 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.741622 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260927 07:09:27.742337 11592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.174.124@KRBTEST.COM
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.124@KRBTEST.COM for oryx/127.4.174.125@KRBTEST.COM
I20260927 07:09:27.748525 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.740122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.4.174.124:45073)
0927 07:09:27.740298 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.740303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.740329 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:09:27.740352 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.740354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.740367 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.740427 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.741094 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.742091 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.743033 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.743472 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.743563 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.745802 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.745808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.745809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.745815 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.747396 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.747730 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.747733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.747733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.747818 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.747992 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.747992 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.747992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.748145 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.748263 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.748350 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.748422 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260927 07:09:27.748817 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.742619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.0.0.1:51830)
0927 07:09:27.742654 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.742663 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.742693 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.742818 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.742819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.742917 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.742962 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.742963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.743835 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.743836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.744404 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.744406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.744745 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.744746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.744812 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.745131 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.745136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.746173 (+ 1037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.747764 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.747765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.747765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.747988 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.748170 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.748170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.748170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.748229 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.748491 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.748492 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.748544 (+ 52us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.748629 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.748729 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 07:09:27.748829 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.742668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.4.174.125:37471)
0927 07:09:27.742714 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.742715 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.742723 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:27.742745 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.742746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.742755 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.742816 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.743021 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.743738 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.744480 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.744690 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.744742 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.746254 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.746255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.746255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.746261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.747694 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.748049 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.748050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.748050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.748102 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.748285 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.748286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.748286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.748409 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.748472 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.748660 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.748743 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:09:27.749346 11592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.124@KRBTEST.COM for oryx/127.4.174.124@KRBTEST.COM
I20260927 07:09:27.754542 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.749564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:45073 (local address 127.0.0.1:54686)
0927 07:09:27.749602 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.749611 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.749645 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.749729 (+ 84us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.749729 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.749825 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.749851 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.749852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.750391 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.750392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.750927 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.750929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.751216 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.751216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.751256 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.751573 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.751577 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.752672 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.753851 (+ 1179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.753851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.753851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.753995 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.754128 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.754129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.754129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.754172 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.754320 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.754320 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.754350 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.754392 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.754458 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260927 07:09:27.754634 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.749605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.4.174.124:45073)
0927 07:09:27.749651 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.749652 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.749660 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:27.749679 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.749679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.749687 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.749725 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.749894 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.750337 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.750984 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.751165 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.751210 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.752753 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.752754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.752754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.752759 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.753787 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.754047 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.754047 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.754048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.754087 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.754204 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.754205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.754205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.754283 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.754324 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.754406 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.754496 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":31}
I20260927 07:09:27.755837 11592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Bootstrap starting.
I20260927 07:09:27.756582 11592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:27.756803 11592 log.cc:824] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:27.757525 11592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: No bootstrap required, opened a new log
I20260927 07:09:27.759044 11592 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } }
I20260927 07:09:27.759186 11592 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:27.759207 11592 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8260de92b0b4045a00d10ee509503c5, State: Initialized, Role: FOLLOWER
I20260927 07:09:27.759296 11592 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } }
I20260927 07:09:27.759564 11592 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:27.759523 11597 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } } }
I20260927 07:09:27.759625 11597 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 07:09:27.761093 11608 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:09:27.761252 11608 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:09:27.763232 11524 master_runner.cc:428] Detected that this master f8260de92b0b4045a00d10ee509503c5 is joining an existing cluster
I20260927 07:09:27.763273 11524 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.174.124:45073
I20260927 07:09:27.770618 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.764183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:53630)
0927 07:09:27.764309 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.764320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.764367 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.764601 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.764602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.764867 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.764946 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.764948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.765855 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.765856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.766781 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.766783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.767265 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.767266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.767323 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.767760 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.767765 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.768091 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.769692 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.769693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.769693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.769850 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.770054 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.770055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.770055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.770103 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.770346 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.770347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.770389 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.770433 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.770503 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":31,"thread_start_us":70,"threads_started":1}
I20260927 07:09:27.770659 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.764362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.4.174.126:33097)
0927 07:09:27.764456 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.764458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.764466 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:27.764488 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.764489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.764500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.764554 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.765037 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.765779 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.766871 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.767160 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.767193 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.768173 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.768175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.768175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.768183 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.769597 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.769919 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.769920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.769920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.769994 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.770156 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.770157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.770157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.770292 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.770360 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.770461 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.770534 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492966, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.126@KRBTEST.COM for oryx/127.4.174.124@KRBTEST.COM
I20260927 07:09:27.799933 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 10934
I20260927 07:09:27.806469 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:44313
--local_ip_for_outbound_sockets=127.4.174.65
--tserver_master_addrs=127.4.174.126:33097
--webserver_port=46331
--webserver_interface=127.4.174.65
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab
--principal=oryx/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471,127.4.174.124:45073 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:09:27.806639 11616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.764185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:45073 (local address 127.0.0.1:54702)
0927 07:09:27.764531 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.764540 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.764592 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.764702 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.764702 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.764911 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.764964 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.764965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.765849 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.765849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.766781 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.766783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.767260 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.767261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.767323 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.803462 (+ 36139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.803469 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.803925 (+ 456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.805689 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.805690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.805690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.805880 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.806024 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.806025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.806025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.806066 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.806309 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.806310 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.806348 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.806399 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.806499 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":34,"thread_start_us":80,"threads_started":1}
I20260927 07:09:27.806867 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.764411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.4.174.124:45073)
0927 07:09:27.764446 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.764448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.764579 (+ 131us) server_negotiation.cc:407] Connection header received
0927 07:09:27.764630 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.764631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.764639 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.764696 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.765029 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.765767 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.766864 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.767135 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.767180 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.804079 (+ 36899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.804081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.804081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.804088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.805623 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.805940 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.805940 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.805941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.805978 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.806112 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.806113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.806113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.806236 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.806289 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.806737 (+ 448us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.806801 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:09:27.807685 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.772593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:45073 (local address 127.0.0.1:54704)
0927 07:09:27.772633 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.772643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.772685 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.772927 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.772929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.773067 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.773102 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.773103 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.773717 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.773718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.774218 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.774219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.774545 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.774546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.774639 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.774981 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.774986 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.776315 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.805069 (+ 28754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.805071 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.805071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.805311 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.805545 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.805546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.805546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.805611 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.807359 (+ 1748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.807361 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.807416 (+ 55us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.807465 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.807592 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260927 07:09:27.808043 11617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.772667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.4.174.124:45073)
0927 07:09:27.772815 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.772817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.772822 (+ 5us) server_negotiation.cc:407] Connection header received
0927 07:09:27.772857 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.772857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.772865 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.772917 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.773179 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.773650 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.774285 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.774492 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.774542 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.776411 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.776413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.776413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.776420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.804966 (+ 28546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.805400 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.805401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.805402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.805458 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.805676 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.805677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.805677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.807000 (+ 1323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.807803 (+ 803us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.807844 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.807925 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260927 07:09:27.808153 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.764504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.4.174.125:37471)
0927 07:09:27.764544 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.764546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.764839 (+ 293us) server_negotiation.cc:407] Connection header received
0927 07:09:27.764933 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.764934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.764945 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.764971 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.765347 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.766098 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.767180 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.767463 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.767502 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.804924 (+ 37422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.804926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.804926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.804937 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.806713 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.807388 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.807390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.807390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.807448 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.807651 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.807651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.807652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.807764 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.807800 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.807990 (+ 190us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.808068 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:27.808445 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.764372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.0.0.1:51836)
0927 07:09:27.764781 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.764790 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.764865 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.765024 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.765025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.765200 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.765258 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.765260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.766192 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.766192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.767082 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.767085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.767579 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.767579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.767651 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.803909 (+ 36258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.803914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.804777 (+ 863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.807092 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.807093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.807093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.807296 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.807512 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.807512 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.807513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.807574 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.807839 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.807839 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.807891 (+ 52us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.808258 (+ 367us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.808351 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":58,"threads_started":1}
I20260927 07:09:27.808421 10872 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.174.124:45073: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f8260de92b0b4045a00d10ee509503c5"
member_type: NON_VOTER
last_known_addr {
host: "127.4.174.124"
port: 45073
}
attrs {
promote: true
}
}
dest_uuid: "bfd23aea91a3461894fdd3149d49b4d7"
cas_config_opid_index: 5
I20260927 07:09:27.808667 10872 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } }
I20260927 07:09:27.809127 11619 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f8260de92b0b4045a00d10ee509503c5 (127.4.174.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.809643 11566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:27.810597 11566 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 FOLLOWER]: Refusing update from remote peer bfd23aea91a3461894fdd3149d49b4d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 07:09:27.810868 11432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:27.813946 11619 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.174.124:45073
I20260927 07:09:27.814042 11432 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.814118 11432 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:27.814363 11524 master.cc:537] Master@127.4.174.124:45073 shutting down...
I20260927 07:09:27.815037 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.809245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.0.0.1:51840)
0927 07:09:27.811271 (+ 2026us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.811282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.811321 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.811532 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.811533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.811535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:27.811582 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.811584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.812477 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.812479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.813680 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.813683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.814728 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.814729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.814836 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.814867 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.814916 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.814958 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2009}
I20260927 07:09:27.815310 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.809327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.4.174.125:37471)
0927 07:09:27.809360 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.809362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.811367 (+ 2005us) server_negotiation.cc:407] Connection header received
0927 07:09:27.811389 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.811390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.811414 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.811467 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:27.811650 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.812379 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.813803 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.814402 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.814461 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.814574 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.815196 (+ 622us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.815247 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:09:27.815510 11480 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 LEARNER]: Refusing update from remote peer bfd23aea91a3461894fdd3149d49b4d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 07:09:27.815711 11619 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260927 07:09:27.816267 11480 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f8260de92b0b4045a00d10ee509503c5 (127.4.174.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.817370 11510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd23aea91a3461894fdd3149d49b4d7. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.817463 11510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: This master's current role is: LEARNER
I20260927 07:09:27.817497 11510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.817554 11510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: This master's current role is: LEARNER
I20260927 07:09:27.817588 11566 raft_consensus.cc:858] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } }
W20260927 07:09:27.817651 11566 raft_consensus.cc:1565] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } }
I20260927 07:09:27.818071 11566 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 65020500fc70441b97bf3547b2ed780b (127.4.174.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } }
I20260927 07:09:27.818297 11622 mvcc.cc:205] Tried to move back new op lower bound from 7333859192267321344 to 7333859192161632256. Current Snapshot: MvccSnapshot[applied={T|T < 7333859192170848256}]
I20260927 07:09:27.818867 11619 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: attempting to promote NON_VOTER 65020500fc70441b97bf3547b2ed780b to VOTER
I20260927 07:09:27.819078 11619 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } }
I20260927 07:09:27.819331 11597 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd23aea91a3461894fdd3149d49b4d7. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } }
I20260927 07:09:27.819414 11597 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 07:09:27.819454 11597 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } }
I20260927 07:09:27.819495 11597 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 07:09:27.819540 11597 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: NON_VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: true } } }
I20260927 07:09:27.819583 11597 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 07:09:27.819770 11480 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 LEARNER]: Refusing update from remote peer bfd23aea91a3461894fdd3149d49b4d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 07:09:27.819952 11431 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260927 07:09:27.820050 10856 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 -> Peer f8260de92b0b4045a00d10ee509503c5 (127.4.174.124:45073): Couldn't send request to peer f8260de92b0b4045a00d10ee509503c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 07:09:27.820842 11431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 65020500fc70441b97bf3547b2ed780b (127.4.174.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.821596 11431 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.821684 11431 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:27.821735 11431 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.821779 11431 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:27.821880 11480 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 65020500fc70441b97bf3547b2ed780b (127.4.174.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.822625 11621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.822707 11621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:09:27.823808 11524 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260927 07:09:27.823923 11524 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260927 07:09:27.823967 11524 tablet_replica.cc:330] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: stopping tablet replica
I20260927 07:09:27.837061 11524 master.cc:552] Master@127.4.174.124:45073 shutdown complete.
I20260927 07:09:27.837108 11524 master_runner.cc:305] Clearing existing system tablet
I20260927 07:09:27.837231 11524 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.837273 11524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.837888 11524 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.838358 11630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.838526 11524 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.838595 11524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.838619 11524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
uuid: "f8260de92b0b4045a00d10ee509503c5"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "a05328aaea8b2cf9e5b136b426933de5"
server_key_iv: "1ba0ab993e860a458f1a71d2754aa6c4"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.839262 11524 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:27.840613 11524 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 07:09:27.840680 11524 log.cc:1196] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260927 07:09:27.840899 11524 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Deleting consensus metadata
I20260927 07:09:27.841003 11524 master_runner.cc:315] Copying system tablet from 127.4.174.126:33097
I20260927 07:09:27.841571 11524 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: tablet copy: Beginning tablet copy session from remote peer at address 127.4.174.126:33097
I20260927 07:09:27.847435 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.841890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.4.174.126:33097)
0927 07:09:27.841936 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.841938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.842049 (+ 111us) server_negotiation.cc:407] Connection header received
0927 07:09:27.842117 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.842118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.842127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.842169 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.842497 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.843053 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.843860 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.844058 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.844113 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.845043 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.845045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.845045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.845053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.846354 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.846760 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.846761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.846762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.846814 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.846970 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.846971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.846971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.847081 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.847137 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.847266 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.847343 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:27.847452 11636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.841841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:53640)
0927 07:09:27.841979 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.841989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.842049 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.842241 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.842241 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.842382 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.842431 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.842433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.843138 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.843139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.843716 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.843720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.844124 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.844125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.844219 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.844665 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.844670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.844963 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.846427 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.846428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.846429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.846693 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.846863 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.846864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.846865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.846923 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.847134 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.847135 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.847179 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.847239 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.847344 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260927 07:09:27.848037 10892 tablet_copy_service.cc:140] P bfd23aea91a3461894fdd3149d49b4d7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f8260de92b0b4045a00d10ee509503c5 ({username='oryx', principal='oryx/127.4.174.124@KRBTEST.COM'} at 127.0.0.1:53640)
I20260927 07:09:27.848104 10892 tablet_copy_service.cc:161] P bfd23aea91a3461894fdd3149d49b4d7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f8260de92b0b4045a00d10ee509503c5 at {username='oryx', principal='oryx/127.4.174.124@KRBTEST.COM'} at 127.0.0.1:53640: session id = f8260de92b0b4045a00d10ee509503c5-00000000000000000000000000000000
I20260927 07:09:27.848567 10892 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 07:09:27.849006 11524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 07:09:27.849911 11524 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: tablet copy: Starting download of 0 data blocks...
I20260927 07:09:27.850042 11524 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: tablet copy: Starting download of 1 WAL segments...
I20260927 07:09:27.850693 11524 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 07:09:27.851413 10892 tablet_copy_service.cc:342] P bfd23aea91a3461894fdd3149d49b4d7: Request end of tablet copy session f8260de92b0b4045a00d10ee509503c5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.174.124@KRBTEST.COM'} at 127.0.0.1:53640
I20260927 07:09:27.851464 10892 tablet_copy_service.cc:433] P bfd23aea91a3461894fdd3149d49b4d7: ending tablet copy session f8260de92b0b4045a00d10ee509503c5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f8260de92b0b4045a00d10ee509503c5
I20260927 07:09:27.853727 11524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:27.854863 11640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.855101 11637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.855165 11524 server_base.cc:1034] running on GCE node
W20260927 07:09:27.855222 11638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.855376 11524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:27.855505 11524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:27.856709 11524 hybrid_clock.cc:648] HybridClock initialized: now 1790492967856700 us; error 25 us; skew 500 ppm
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:27.857983 11524 init.cc:377] Logged in from keytab as oryx/127.4.174.124@KRBTEST.COM (short username oryx)
I20260927 07:09:27.858240 11524 webserver.cc:460] Webserver started at http://127.4.174.124:42063/ using document root <none> and password file <none>
I20260927 07:09:27.858349 11524 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.858382 11524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.859010 11524 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.859467 11647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.859583 11524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.859624 11524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
uuid: "f8260de92b0b4045a00d10ee509503c5"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "a05328aaea8b2cf9e5b136b426933de5"
server_key_iv: "1ba0ab993e860a458f1a71d2754aa6c4"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.859683 11524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.882231 11524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:27.883839 11524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:27.886756 11524 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.124:45073
I20260927 07:09:27.886777 11699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.124:45073 every 8 connection(s)
I20260927 07:09:27.887063 11524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
I20260927 07:09:27.887866 11700 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:09:27.888262 11700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Bootstrap starting.
I20260927 07:09:27.889988 11700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:09:27.890174 11700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Bootstrap complete.
I20260927 07:09:27.890362 11700 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } }
I20260927 07:09:27.890434 11700 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f8260de92b0b4045a00d10ee509503c5, State: Initialized, Role: LEARNER
I20260927 07:09:27.890468 11700 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } }
I20260927 07:09:27.890669 11700 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:27.890654 11703 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:27.890848 11703 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: LEARNER
I20260927 07:09:27.892007 11714 catalog_manager.cc:1259] Loaded cluster ID: 841d0267662c4122bf6520a898268acc
I20260927 07:09:27.892278 11714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: loading cluster ID for follower catalog manager: success
I20260927 07:09:27.893296 11714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: acquiring CA information for follower catalog manager: success
I20260927 07:09:27.893555 11714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 07:09:27.927322 11618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:27.927526 11618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:27.927551 11618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:27.928679 11618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:27.928714 11618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:27.928740 11618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:27.930143 11618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab
--principal=oryx/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:44313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=46331
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471,127.4.174.124:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:27.930325 11618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:27.930500 11618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:27.935245 11722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.935298 11719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:27.935352 11720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:27.935374 11618 server_base.cc:1034] running on GCE node
I20260927 07:09:27.935648 11618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:27.935843 11618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:27.936981 11618 hybrid_clock.cc:648] HybridClock initialized: now 1790492967936974 us; error 32 us; skew 500 ppm
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:27.939364 11618 init.cc:377] Logged in from keytab as oryx/127.4.174.65@KRBTEST.COM (short username oryx)
I20260927 07:09:27.939824 11618 webserver.cc:460] Webserver started at http://127.4.174.65:46331/ using document root <none> and password file <none>
I20260927 07:09:27.940024 11618 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:27.940086 11618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:27.941392 11618 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:27.941982 11729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.942154 11618 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:27.942237 11618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea"
format_stamp: "Formatted at 2026-09-27 07:09:26 on dist-test-slave-zw7p"
server_key: "2f8021f43aac8f51eb0a9f6651a1cc09"
server_key_iv: "098a52ad838ad9cc1be0bc0c7872256e"
server_key_version: "encryptionkey@0"
I20260927 07:09:27.942617 11618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:27.956972 11618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:27.959123 11618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:27.959261 11618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:27.959488 11618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:27.959796 11618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:27.959829 11618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.959858 11618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:27.959874 11618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:27.965303 11618 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:44313
I20260927 07:09:27.965358 11842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:44313 every 8 connection(s)
I20260927 07:09:27.965690 11618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.125@KRBTEST.COM
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.124@KRBTEST.COM
Sep 27 07:09:27 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492967, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:27.974591 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 11618
I20260927 07:09:27.974746 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 11068
I20260927 07:09:27.976459 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.966363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:36621 (local address 127.4.174.125:37471)
0927 07:09:27.966407 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.966409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.966853 (+ 444us) server_negotiation.cc:407] Connection header received
0927 07:09:27.967753 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.967755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.967768 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.967822 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.968648 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.969442 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970320 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.970641 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970705 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.973172 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.973174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.973174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.973183 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.974724 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.975419 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.975421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.975421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.975483 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.975763 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.975764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.975764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.975903 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.975969 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.976288 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.976367 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:09:27.976593 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.966381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.4.174.65:36621)
0927 07:09:27.966789 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.966798 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.967659 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.967905 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.967910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.968325 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.968547 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.968553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.969538 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.969539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.970243 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.970246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.970716 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.970852 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.971419 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.971429 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.973065 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.974822 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.974829 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.974832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.975343 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.975557 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.975558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.975558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.975704 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.975972 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.975979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.976150 (+ 171us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.976260 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.976400 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":34,"thread_start_us":87,"threads_started":1}
I20260927 07:09:27.977265 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.966812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:38107 (local address 127.4.174.124:45073)
0927 07:09:27.966923 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.966925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.966934 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:27.967754 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.967756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.967771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.967834 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.968649 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.969684 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970535 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.970830 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970896 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.973858 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.973859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.973860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.973867 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.975758 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.976023 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.976024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.976025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.976079 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.976597 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.976598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.976598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.976892 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.976949 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.977127 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.977201 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
I20260927 07:09:27.977278 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.966381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:45073 (local address 127.4.174.65:38107)
0927 07:09:27.966592 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.966639 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.967640 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.967905 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.967910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.968335 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.968547 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.968553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.969764 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.969765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.970466 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.970467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.970905 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.970977 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.971593 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.971597 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.973778 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.975817 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.975818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.975818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.975970 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.976134 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.976134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.976134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.976203 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.976974 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.976975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.977017 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.977084 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.977173 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":6,"thread_start_us":91,"threads_started":1}
I20260927 07:09:27.979607 11845 heartbeater.cc:347] Connected to a master server at 127.4.174.125:37471
I20260927 07:09:27.979717 11845 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:27.979935 11845 heartbeater.cc:511] Master 127.4.174.125:37471 requested a full tablet report, sending...
I20260927 07:09:27.980528 11470 ts_manager.cc:194] Registered new tserver with Master: 6e2ac51ab50e41cd8d637a2cd42d19ea (127.4.174.65:44313)
I20260927 07:09:27.981158 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.966772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:34579 (local address 127.4.174.126:33097)
0927 07:09:27.966924 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:27.966926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:27.967862 (+ 936us) server_negotiation.cc:407] Connection header received
0927 07:09:27.967934 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:27.967935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:27.967945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:27.967990 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:27.968632 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.969126 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970356 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.970678 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970729 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.974167 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:27.974169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:27.974170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:27.974178 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:27.980138 (+ 5960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.980463 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.980464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.980464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.980525 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:27.980696 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:27.980697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:27.980697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:27.980811 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:27.980867 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:27.980999 (+ 132us) server_negotiation.cc:299] Negotiation successful
0927 07:09:27.981072 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260927 07:09:27.981180 11843 heartbeater.cc:347] Connected to a master server at 127.4.174.124:45073
I20260927 07:09:27.981200 11848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:27.966690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:34579)
0927 07:09:27.967809 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:27.967818 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:27.967885 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:27.967994 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:27.967995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:27.968301 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:27.968546 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.968553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.969222 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.969223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.970247 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:27.970250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:27.970776 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:27.970777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:27.970852 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:27.971462 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:27.971466 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:27.974069 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:27.980211 (+ 6142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.980212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.980212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.980397 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.980576 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:27.980576 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:27.980577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:27.980633 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:27.980866 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:27.980866 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:27.980908 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 07:09:27.980968 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:09:27.981066 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"mutex_wait_us":34,"thread_start_us":58,"threads_started":1}
I20260927 07:09:27.981243 11843 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:27.981305 11843 heartbeater.cc:511] Master 127.4.174.124:45073 requested a full tablet report, sending...
I20260927 07:09:27.981415 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:43757
--local_ip_for_outbound_sockets=127.4.174.66
--tserver_master_addrs=127.4.174.126:33097
--webserver_port=36053
--webserver_interface=127.4.174.66
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab
--principal=oryx/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471,127.4.174.124:45073 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:09:27.981765 11844 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33097
I20260927 07:09:27.981817 11844 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:27.981873 11844 heartbeater.cc:511] Master 127.4.174.126:33097 requested a full tablet report, sending...
I20260927 07:09:27.982125 10872 ts_manager.cc:194] Re-registered known tserver with Master: 6e2ac51ab50e41cd8d637a2cd42d19ea (127.4.174.65:44313)
I20260927 07:09:27.982451 10872 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:34579
I20260927 07:09:27.982820 11660 ts_manager.cc:194] Registered new tserver with Master: 6e2ac51ab50e41cd8d637a2cd42d19ea (127.4.174.65:44313)
W20260927 07:09:28.062206 11851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:28.062430 11851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:28.062453 11851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:28.063748 11851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:28.063793 11851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:28.063819 11851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:28.065296 11851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab
--principal=oryx/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:43757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=36053
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471,127.4.174.124:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:28.065531 11851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:28.065773 11851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:28.070667 11857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:28.070791 11851 server_base.cc:1034] running on GCE node
W20260927 07:09:28.070658 11859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:28.070699 11856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:28.071142 11851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:28.071373 11851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:28.072535 11851 hybrid_clock.cc:648] HybridClock initialized: now 1790492968072519 us; error 27 us; skew 500 ppm
Sep 27 07:09:28 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492968, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:28.074987 11851 init.cc:377] Logged in from keytab as oryx/127.4.174.66@KRBTEST.COM (short username oryx)
I20260927 07:09:28.075436 11851 webserver.cc:460] Webserver started at http://127.4.174.66:36053/ using document root <none> and password file <none>
I20260927 07:09:28.075692 11851 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:28.075768 11851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:28.077227 11851 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:28.077895 11866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:28.078104 11851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:28.078193 11851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "f8d46f0507454efe97b2840b0a6016ca"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "48ab42988717542295362d896bc8bc2b"
server_key_iv: "9fee7fec9bd323b30be80d9a5989faec"
server_key_version: "encryptionkey@0"
I20260927 07:09:28.078538 11851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:28.109144 11851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:28.111009 11851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:28.111148 11851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:28.111361 11851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:28.111653 11851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:28.111687 11851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:28.111716 11851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:28.111739 11851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:28.117266 11851 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:43757
I20260927 07:09:28.117318 11979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:43757 every 8 connection(s)
I20260927 07:09:28.117643 11851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:09:28 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492968, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.125@KRBTEST.COM
Sep 27 07:09:28 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492968, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
Sep 27 07:09:28 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492968, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.124@KRBTEST.COM
I20260927 07:09:28.126351 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 11851
I20260927 07:09:28.126528 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 11202
I20260927 07:09:28.127558 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.118276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:60769 (local address 127.4.174.126:33097)
0927 07:09:28.118365 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.118367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.118634 (+ 267us) server_negotiation.cc:407] Connection header received
0927 07:09:28.119184 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.119186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.119199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.119249 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.119939 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.120739 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.121758 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.121999 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.122053 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.124913 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.124915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.124915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.124923 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.126265 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.126643 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.126644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.126645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.126700 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.126929 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.126929 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.126930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.127053 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.127094 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.127389 (+ 295us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.127469 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:09:28.127739 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.118307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:60769)
0927 07:09:28.118529 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.118539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.119065 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.119323 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.119326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.119623 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:28.119844 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.119849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.120817 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.120818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.121684 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.121685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.122102 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.122102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.122187 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.122588 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:28.122593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:28.124833 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:28.126335 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.126336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.126337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.126525 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.126760 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.126761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.126761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.126877 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.127120 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:28.127125 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:28.127245 (+ 120us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.127337 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.127496 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":46,"thread_start_us":74,"threads_started":1}
I20260927 07:09:28.127872 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.118307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:45073 (local address 127.4.174.66:37937)
0927 07:09:28.118489 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.118531 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.119103 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.119678 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.119678 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.119823 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:28.119868 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.119870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.120554 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.120555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.121592 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.121595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.122047 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.122048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.122110 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.122910 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:28.122914 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:28.125084 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:28.126907 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.126908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.126909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.127090 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.127296 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.127297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.127297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.127347 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.127584 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:28.127585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:28.127624 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.127678 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.127770 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260927 07:09:28.128149 11981 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33097
I20260927 07:09:28.128207 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.118278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:37937 (local address 127.4.174.124:45073)
0927 07:09:28.118321 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.118323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.118631 (+ 308us) server_negotiation.cc:407] Connection header received
0927 07:09:28.119215 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.119216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.119227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.119270 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.119946 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.120448 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.121685 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.121974 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.122038 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.125193 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.125194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.125194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.125203 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.126828 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.127172 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.127173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.127174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.127229 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.127401 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.127402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.127402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.127516 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.127563 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.128025 (+ 462us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.128123 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:09:28.128239 11981 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:28.128423 11981 heartbeater.cc:511] Master 127.4.174.126:33097 requested a full tablet report, sending...
I20260927 07:09:28.128782 10872 ts_manager.cc:194] Re-registered known tserver with Master: f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757)
I20260927 07:09:28.129148 10872 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:60769
I20260927 07:09:28.131711 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.118733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:35429 (local address 127.4.174.125:37471)
0927 07:09:28.118787 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.118789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.118798 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:28.119175 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.119176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.119188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.119219 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.119965 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.120739 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.121573 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.121785 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.121847 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.124161 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.124162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.124163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.124170 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.125675 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.126094 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.126095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.126096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.126163 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.131254 (+ 5091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.131255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.131255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.131375 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.131426 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.131548 (+ 122us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.131623 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 07:09:28.131814 11980 heartbeater.cc:347] Connected to a master server at 127.4.174.124:45073
I20260927 07:09:28.131860 11980 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:28.131881 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:41547
--local_ip_for_outbound_sockets=127.4.174.67
--tserver_master_addrs=127.4.174.126:33097
--webserver_port=39381
--webserver_interface=127.4.174.67
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab
--principal=oryx/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471,127.4.174.124:45073 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:09:28.131925 11980 heartbeater.cc:511] Master 127.4.174.124:45073 requested a full tablet report, sending...
I20260927 07:09:28.131755 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.118645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.4.174.66:35429)
0927 07:09:28.118745 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.118753 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.119081 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.119335 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.119336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.119634 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:28.119846 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.119853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.120848 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.120849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.121497 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.121498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.121856 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.121857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.121982 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.122431 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:28.122439 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:28.124058 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:28.125763 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.125771 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.125774 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.126034 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.131100 (+ 5066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.131101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.131102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.131179 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.131430 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:28.131431 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:28.131471 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.131532 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.131646 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":64,"threads_started":1}
I20260927 07:09:28.132335 11982 heartbeater.cc:347] Connected to a master server at 127.4.174.125:37471
I20260927 07:09:28.132370 11982 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:28.132408 11982 heartbeater.cc:511] Master 127.4.174.125:37471 requested a full tablet report, sending...
I20260927 07:09:28.132683 11470 ts_manager.cc:194] Registered new tserver with Master: f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757)
I20260927 07:09:28.133221 11660 ts_manager.cc:194] Registered new tserver with Master: f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757)
W20260927 07:09:28.211344 11987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:28.211555 11987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:28.211577 11987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:28.212687 11987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:28.212718 11987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:28.212742 11987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:28.214195 11987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab
--principal=oryx/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:41547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=39381
--tserver_master_addrs=127.4.174.126:33097,127.4.174.125:37471,127.4.174.124:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:28.214432 11987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:28.214660 11987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:28.219290 11993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:28.219357 11992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:28.219434 11987 server_base.cc:1034] running on GCE node
W20260927 07:09:28.219295 11995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:28.219730 11987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:28.219985 11987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:28.221136 11987 hybrid_clock.cc:648] HybridClock initialized: now 1790492968221141 us; error 55 us; skew 500 ppm
Sep 27 07:09:28 dist-test-slave-zw7p krb5kdc[10827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492968, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:28.223567 11987 init.cc:377] Logged in from keytab as oryx/127.4.174.67@KRBTEST.COM (short username oryx)
I20260927 07:09:28.223997 11987 webserver.cc:460] Webserver started at http://127.4.174.67:39381/ using document root <none> and password file <none>
I20260927 07:09:28.224205 11987 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:28.224336 11987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:28.225632 11987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:28.226241 12002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:28.226402 11987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:28.226491 11987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "d64dde1fc3b94d059545eb443a4e85c9"
format_stamp: "Formatted at 2026-09-27 07:09:27 on dist-test-slave-zw7p"
server_key: "499f86aa09fc1ffd63f12040e7f5d989"
server_key_iv: "d4d6c337858d4708b557f2cf8b72e752"
server_key_version: "encryptionkey@0"
I20260927 07:09:28.226861 11987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:28.245471 11987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:28.247880 11987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:28.248039 11987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:28.248252 11987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:28.248602 11987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:28.248642 11987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:28.248674 11987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:28.248692 11987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:28.254050 11987 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:41547
I20260927 07:09:28.254088 12115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:41547 every 8 connection(s)
I20260927 07:09:28.254420 11987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:28.256592 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 11987
Sep 27 07:09:28 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492968, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.125@KRBTEST.COM
Sep 27 07:09:28 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492968, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
Sep 27 07:09:28 dist-test-slave-zw7p krb5kdc[10827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492968, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.124@KRBTEST.COM
I20260927 07:09:28.268133 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.258047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.4.174.126:33097)
0927 07:09:28.259030 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.259032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.259038 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:09:28.259056 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.259056 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.259066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.259859 (+ 793us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.259893 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.260630 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.262455 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.262745 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.262796 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.264824 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.264825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.264826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.264832 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.266278 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.266674 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.266674 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.266675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.266731 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.266928 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.266929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.266929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.267050 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.267900 (+ 850us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.267958 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.268032 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":959,"thread_start_us":87,"threads_started":1}
I20260927 07:09:28.268911 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.257701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.4.174.124:45073)
0927 07:09:28.258689 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.258691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.258699 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:28.258720 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.258721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.258732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.258793 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.260952 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.261611 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.262652 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.262934 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.263069 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.264176 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.264177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.264178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.264185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.267233 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.267626 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.267627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.267627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.267685 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.267897 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.267898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.267898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.268030 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.268678 (+ 648us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.268720 (+ 42us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.268797 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967,"thread_start_us":55,"threads_started":1}
I20260927 07:09:28.269378 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.255407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.4.174.67:58281)
0927 07:09:28.256426 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.256435 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.256504 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.256679 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.256680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.257146 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:28.257426 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.257431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.258160 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.258161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.259299 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.259301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.259844 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.259845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.259976 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.260820 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:28.260829 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:28.264014 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:28.266822 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.266828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.266831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.267119 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.267307 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.267307 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.267308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.267439 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.268128 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:28.268134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:28.268295 (+ 161us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.268623 (+ 328us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.268792 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":12,"thread_start_us":54,"threads_started":1}
I20260927 07:09:28.270710 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.255481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:58281 (local address 127.4.174.125:37471)
0927 07:09:28.255524 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.255526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.256477 (+ 951us) server_negotiation.cc:407] Connection header received
0927 07:09:28.256565 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.256566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.256577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.256618 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.257526 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.258073 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.259384 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.259700 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.259777 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.264120 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.264121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.264122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.264129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.266748 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.267190 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.267191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.267191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.267246 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.267899 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.267900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.267901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.268029 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.268088 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.270513 (+ 2425us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.270618 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:09:28.270818 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.255108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:48859 (local address 127.4.174.124:45073)
0927 07:09:28.255157 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.255159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.255424 (+ 265us) server_negotiation.cc:407] Connection header received
0927 07:09:28.256446 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.256448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.256461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.256510 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.258813 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.259571 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.260625 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.260906 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.260949 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.265324 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.265326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.265326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.265333 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.266867 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.269947 (+ 3080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.269948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.269949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.270017 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.270238 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.270239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.270239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.270363 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.270407 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.270657 (+ 250us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.270731 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:28.271212 12118 heartbeater.cc:347] Connected to a master server at 127.4.174.125:37471
I20260927 07:09:28.271307 12118 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:28.271492 12118 heartbeater.cc:511] Master 127.4.174.125:37471 requested a full tablet report, sending...
I20260927 07:09:28.269747 10872 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 07:09:28.271876 11470 ts_manager.cc:194] Registered new tserver with Master: d64dde1fc3b94d059545eb443a4e85c9 (127.4.174.67:41547)
W20260927 07:09:28.271974 10872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:28.272362 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.255124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:40823)
0927 07:09:28.255306 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.255343 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.256284 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.256572 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.256576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.257132 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:28.257426 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.257431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.258809 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.258810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.259683 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.259685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.260418 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.260419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.260481 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.260967 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:28.260972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:28.264880 (+ 3908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:28.271020 (+ 6140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.271021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.271021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.271378 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.271617 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.271618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.271618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.271674 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.272062 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:28.272063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:28.272102 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.272167 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.272254 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":8,"thread_start_us":90,"threads_started":1}
I20260927 07:09:28.272459 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.258306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.4.174.125:37471)
0927 07:09:28.258979 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.258981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.265322 (+ 6341us) server_negotiation.cc:407] Connection header received
0927 07:09:28.265347 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.265348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.265360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.265420 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.266021 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.266778 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.267772 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.268048 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.268106 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.269435 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.269436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.269437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.269443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.271053 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.271440 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.271441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.271441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.271514 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.271721 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.271722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.271722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.271842 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.271930 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.272298 (+ 368us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.272370 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":55,"threads_started":1}
I20260927 07:09:28.272789 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.255172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:40823 (local address 127.4.174.126:33097)
0927 07:09:28.255210 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.255212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.255422 (+ 210us) server_negotiation.cc:407] Connection header received
0927 07:09:28.256373 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.256375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.256387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.256447 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.258169 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.258732 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.259758 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.260045 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.260102 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.264991 (+ 4889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.264993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.264993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.265001 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.270407 (+ 5406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.271457 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.271458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.271458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.271522 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.271745 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.271746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.271746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.271858 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.271917 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.272624 (+ 707us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.272699 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:09:28.273277 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.255117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:45073 (local address 127.4.174.67:48859)
0927 07:09:28.255361 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.255371 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.256270 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.256573 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.256577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.257122 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:28.257426 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.257432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.259762 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.259762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.260536 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.260538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.261034 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.261035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.261101 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.261409 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:28.261414 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:28.265231 (+ 3817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:28.269638 (+ 4407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.269639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.269639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.269852 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.270097 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:28.270098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:28.270098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:28.270160 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:28.270459 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:28.270460 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:28.270504 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.271111 (+ 607us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.271227 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":27,"thread_start_us":101,"threads_started":1}
I20260927 07:09:28.273720 12117 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33097
I20260927 07:09:28.273779 12117 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:28.273849 12117 heartbeater.cc:511] Master 127.4.174.126:33097 requested a full tablet report, sending...
I20260927 07:09:28.273944 12116 heartbeater.cc:347] Connected to a master server at 127.4.174.124:45073
I20260927 07:09:28.273996 12116 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:28.274065 12116 heartbeater.cc:511] Master 127.4.174.124:45073 requested a full tablet report, sending...
I20260927 07:09:28.274194 10871 ts_manager.cc:194] Re-registered known tserver with Master: d64dde1fc3b94d059545eb443a4e85c9 (127.4.174.67:41547)
I20260927 07:09:28.274348 11660 ts_manager.cc:194] Registered new tserver with Master: d64dde1fc3b94d059545eb443a4e85c9 (127.4.174.67:41547)
I20260927 07:09:28.274596 10871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:40823
I20260927 07:09:28.280316 12137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.276491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43757 (local address 127.0.0.1:33144)
0927 07:09:28.276615 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.276622 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.276660 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.277073 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.277074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.277076 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:28.277128 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.277129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.278439 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.278440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.279192 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.279195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.280069 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.280071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.280162 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.280167 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.280195 (+ 28us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.280235 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260927 07:09:28.280473 12140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.276694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.4.174.67:41547)
0927 07:09:28.276865 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.276870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.276882 (+ 12us) server_negotiation.cc:407] Connection header received
0927 07:09:28.276922 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.276924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.277048 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.277282 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:28.277326 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.277968 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.279431 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.280108 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.280169 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.280245 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.280368 (+ 123us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.280404 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260927 07:09:28.280449 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.276419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:44313 (local address 127.0.0.1:53432)
0927 07:09:28.276469 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.276480 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.276520 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.277073 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.277074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.277075 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:28.277121 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.277123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.278418 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.278419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.279464 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.279466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.280251 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.280252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.280300 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.280302 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.280331 (+ 29us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.280372 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":28}
I20260927 07:09:28.280725 12141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.276726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.4.174.66:43757)
0927 07:09:28.276915 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.276919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.276928 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:28.276960 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.276962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.276975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.277193 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:28.277249 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.278336 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.279267 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.279971 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.280483 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.280576 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.280600 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.280641 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260927 07:09:28.281008 12138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.276505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:41547 (local address 127.0.0.1:52438)
0927 07:09:28.276719 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.276729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.276781 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.277155 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.277156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.277158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:28.277206 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.277208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.278057 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.278058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.279353 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.279355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.280221 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.280223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.280285 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.280290 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.280349 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.280918 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260927 07:09:28.281268 12139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.276508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.4.174.65:44313)
0927 07:09:28.276900 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.276905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.276915 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:28.276955 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.276957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.276973 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.277194 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:28.277254 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.278337 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.279542 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.280148 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.280587 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.280990 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.281022 (+ 32us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.281071 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":87,"threads_started":1}
I20260927 07:09:28.281355 11914 tablet_service.cc:1467] Processing CreateTablet for tablet 41097523b29842fabe68c36f64c25bdf (DEFAULT_TABLE table=test-table [id=96800a62e8ad4af5ac2ba21d0e87e365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:28.281730 11914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41097523b29842fabe68c36f64c25bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:28.281791 12050 tablet_service.cc:1467] Processing CreateTablet for tablet 41097523b29842fabe68c36f64c25bdf (DEFAULT_TABLE table=test-table [id=96800a62e8ad4af5ac2ba21d0e87e365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:28.282104 12050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41097523b29842fabe68c36f64c25bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:28.283209 11777 tablet_service.cc:1467] Processing CreateTablet for tablet 41097523b29842fabe68c36f64c25bdf (DEFAULT_TABLE table=test-table [id=96800a62e8ad4af5ac2ba21d0e87e365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:28.283596 11777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41097523b29842fabe68c36f64c25bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:28.284792 12142 tablet_bootstrap.cc:492] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Bootstrap starting.
I20260927 07:09:28.285403 12142 tablet_bootstrap.cc:654] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:28.285550 12143 tablet_bootstrap.cc:492] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Bootstrap starting.
I20260927 07:09:28.285964 12142 log.cc:824] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:28.286159 12143 tablet_bootstrap.cc:654] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:28.286398 12143 log.cc:824] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:28.286868 12142 tablet_bootstrap.cc:492] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: No bootstrap required, opened a new log
I20260927 07:09:28.286931 12142 ts_tablet_manager.cc:1397] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:28.286936 12146 tablet_bootstrap.cc:492] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Bootstrap starting.
I20260927 07:09:28.287513 12146 tablet_bootstrap.cc:654] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:28.287802 12146 log.cc:824] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:28.288560 12146 tablet_bootstrap.cc:492] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: No bootstrap required, opened a new log
I20260927 07:09:28.288635 12146 ts_tablet_manager.cc:1397] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:28.288787 12142 raft_consensus.cc:348] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.288951 12142 raft_consensus.cc:374] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:28.288985 12142 raft_consensus.cc:729] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8d46f0507454efe97b2840b0a6016ca, State: Initialized, Role: FOLLOWER
I20260927 07:09:28.289101 12142 consensus_queue.cc:260] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.289378 11981 heartbeater.cc:503] Master 127.4.174.126:33097 was elected leader, sending a full tablet report...
I20260927 07:09:28.289561 12142 ts_tablet_manager.cc:1428] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:09:28.290403 12143 tablet_bootstrap.cc:492] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: No bootstrap required, opened a new log
I20260927 07:09:28.290473 12143 ts_tablet_manager.cc:1397] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260927 07:09:28.291718 12146 raft_consensus.cc:348] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.291877 12146 raft_consensus.cc:374] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:28.291914 12146 raft_consensus.cc:729] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2ac51ab50e41cd8d637a2cd42d19ea, State: Initialized, Role: FOLLOWER
I20260927 07:09:28.292026 12146 consensus_queue.cc:260] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.292130 12143 raft_consensus.cc:348] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.292266 12146 ts_tablet_manager.cc:1428] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260927 07:09:28.292284 12143 raft_consensus.cc:374] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:28.292313 11844 heartbeater.cc:503] Master 127.4.174.126:33097 was elected leader, sending a full tablet report...
I20260927 07:09:28.292320 12143 raft_consensus.cc:729] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64dde1fc3b94d059545eb443a4e85c9, State: Initialized, Role: FOLLOWER
I20260927 07:09:28.292416 12143 consensus_queue.cc:260] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.292652 12143 ts_tablet_manager.cc:1428] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:28.292951 12117 heartbeater.cc:503] Master 127.4.174.126:33097 was elected leader, sending a full tablet report...
I20260927 07:09:28.344794 12138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.342152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:45073 (local address 127.0.0.1:54734)
0927 07:09:28.342216 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.342226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.342266 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.342451 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.342452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.342454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:28.342527 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.342529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.343197 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.343198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.343937 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.343939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.344495 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.344496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.344590 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.344595 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.344645 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.344720 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 07:09:28.344828 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.342257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.4.174.124:45073)
0927 07:09:28.342320 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.342322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.342330 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:28.342357 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.342358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.342385 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.342441 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:28.342618 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.343120 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.344025 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.344420 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.344491 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.344589 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.344682 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.344749 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:09:28.345121 11668 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260927 07:09:28.345485 11703 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd23aea91a3461894fdd3149d49b4d7. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:28.345609 11703 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: LEARNER
W20260927 07:09:28.368891 11983 tablet.cc:2369] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:28.464250 12150 raft_consensus.cc:482] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:28.464365 12150 raft_consensus.cc:504] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.464740 12150 leader_election.cc:288] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e2ac51ab50e41cd8d637a2cd42d19ea (127.4.174.65:44313), f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757)
W20260927 07:09:28.466535 11846 tablet.cc:2369] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:28.467597 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.464936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43757 (local address 127.4.174.67:49339)
0927 07:09:28.464987 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.464996 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.465070 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.465195 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.465196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.465197 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:28.465250 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.465251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.465837 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.465838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.466629 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.466631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.467342 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.467343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.467415 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.467419 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.467478 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.467514 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":6}
I20260927 07:09:28.467638 12141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.464990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:49339 (local address 127.4.174.66:43757)
0927 07:09:28.465059 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.465061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.465070 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:28.465125 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.465125 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.465133 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.465183 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:28.465308 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.465760 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.466722 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.467263 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.467334 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.467344 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.467522 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.467570 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 07:09:28.467615 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.464923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:44313 (local address 127.4.174.67:33243)
0927 07:09:28.464979 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:28.464989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:28.465030 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:28.465162 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:28.465163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:28.465165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:28.465237 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.465239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.465844 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.465845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.466702 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:28.466704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.467374 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.467375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:28.467438 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.467440 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:09:28.467487 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 07:09:28.467528 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":46}
I20260927 07:09:28.467638 12139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.464973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:33243 (local address 127.4.174.65:44313)
0927 07:09:28.465015 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.465016 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.465026 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:28.465089 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.465090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.465100 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.465153 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:28.465299 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.465772 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.466774 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.467309 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.467366 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.467373 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.467538 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.467577 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:28.467957 11934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41097523b29842fabe68c36f64c25bdf" candidate_uuid: "d64dde1fc3b94d059545eb443a4e85c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d46f0507454efe97b2840b0a6016ca" is_pre_election: true
I20260927 07:09:28.468112 11934 raft_consensus.cc:2454] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64dde1fc3b94d059545eb443a4e85c9 in term 0.
I20260927 07:09:28.468225 11797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41097523b29842fabe68c36f64c25bdf" candidate_uuid: "d64dde1fc3b94d059545eb443a4e85c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" is_pre_election: true
I20260927 07:09:28.468334 12003 leader_election.cc:302] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d64dde1fc3b94d059545eb443a4e85c9, f8d46f0507454efe97b2840b0a6016ca; no voters:
I20260927 07:09:28.468374 11797 raft_consensus.cc:2454] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64dde1fc3b94d059545eb443a4e85c9 in term 0.
I20260927 07:09:28.468483 12150 raft_consensus.cc:2781] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:28.468547 12150 raft_consensus.cc:482] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:28.468571 12150 raft_consensus.cc:3037] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:28.469265 12150 raft_consensus.cc:504] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.469388 12150 leader_election.cc:288] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [CANDIDATE]: Term 1 election: Requested vote from peers 6e2ac51ab50e41cd8d637a2cd42d19ea (127.4.174.65:44313), f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757)
I20260927 07:09:28.469558 11797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41097523b29842fabe68c36f64c25bdf" candidate_uuid: "d64dde1fc3b94d059545eb443a4e85c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea"
I20260927 07:09:28.469573 11934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41097523b29842fabe68c36f64c25bdf" candidate_uuid: "d64dde1fc3b94d059545eb443a4e85c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d46f0507454efe97b2840b0a6016ca"
I20260927 07:09:28.469630 11797 raft_consensus.cc:3037] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:28.469647 11934 raft_consensus.cc:3037] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:28.470306 11797 raft_consensus.cc:2454] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64dde1fc3b94d059545eb443a4e85c9 in term 1.
I20260927 07:09:28.470355 11934 raft_consensus.cc:2454] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64dde1fc3b94d059545eb443a4e85c9 in term 1.
I20260927 07:09:28.470482 12004 leader_election.cc:302] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2ac51ab50e41cd8d637a2cd42d19ea, d64dde1fc3b94d059545eb443a4e85c9; no voters:
I20260927 07:09:28.470618 12150 raft_consensus.cc:2781] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:28.470770 12150 raft_consensus.cc:686] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 1 LEADER]: Becoming Leader. State: Replica: d64dde1fc3b94d059545eb443a4e85c9, State: Running, Role: LEADER
I20260927 07:09:28.470860 12150 consensus_queue.cc:237] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } }
I20260927 07:09:28.471647 10872 catalog_manager.cc:5495] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d64dde1fc3b94d059545eb443a4e85c9 (127.4.174.67). New cstate: current_term: 1 leader_uuid: "d64dde1fc3b94d059545eb443a4e85c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:28.472237 12156 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: NON_VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: true } } }
I20260927 07:09:28.472324 12156 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:28.472838 12156 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: attempting to promote NON_VOTER f8260de92b0b4045a00d10ee509503c5 to VOTER
I20260927 07:09:28.472960 12156 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } }
I20260927 07:09:28.473420 11480 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Refusing update from remote peer bfd23aea91a3461894fdd3149d49b4d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260927 07:09:28.473446 11668 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 LEARNER]: Refusing update from remote peer bfd23aea91a3461894fdd3149d49b4d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260927 07:09:28.473610 12159 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260927 07:09:28.473706 12156 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260927 07:09:28.474273 11432 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f8260de92b0b4045a00d10ee509503c5 (127.4.174.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:28.474496 11668 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f8260de92b0b4045a00d10ee509503c5 (127.4.174.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:28.475040 11432 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:28.475104 12156 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:28.475181 12156 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:28.475260 11480 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f8260de92b0b4045a00d10ee509503c5 (127.4.174.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:28.475304 11703 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:28.475374 11703 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:09:28.475090 12158 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:28.475445 12158 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:28.475127 11432 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:28.476012 12160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfd23aea91a3461894fdd3149d49b4d7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:28.476177 12160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:09:28.504928 12140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.502873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.4.174.67:41547)
0927 07:09:28.502924 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.502926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.502936 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:28.502979 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.502980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.502996 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.503046 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:28.503150 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.503634 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.504298 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.504470 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.504511 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.504619 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:28.504714 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:28.504772 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.504857 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.504874 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260927 07:09:28.505105 12119 tablet.cc:2369] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:28.506621 11797 raft_consensus.cc:1261] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 FOLLOWER]: Refusing update from remote peer d64dde1fc3b94d059545eb443a4e85c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:28.506631 11934 raft_consensus.cc:1261] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 1 FOLLOWER]: Refusing update from remote peer d64dde1fc3b94d059545eb443a4e85c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:28.506943 12154 consensus_queue.cc:1035] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:28.507079 12150 consensus_queue.cc:1035] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:28.510792 12162 mvcc.cc:205] Tried to move back new op lower bound from 7333859199000678400 to 7333859198857351168. Current Snapshot: MvccSnapshot[applied={T|T < 7333859199000678400 or (T in {7333859199000678400})}]
I20260927 07:09:28.510980 12163 mvcc.cc:205] Tried to move back new op lower bound from 7333859199000678400 to 7333859198857351168. Current Snapshot: MvccSnapshot[applied={T|T < 7333859199000678400 or (T in {7333859199000678400})}]
I20260927 07:09:28.511255 12164 mvcc.cc:205] Tried to move back new op lower bound from 7333859199000678400 to 7333859198857351168. Current Snapshot: MvccSnapshot[applied={T|T < 7333859199000678400}]
I20260927 07:09:28.511650 10872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:28.511745 10872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:28.512897 10872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfd23aea91a3461894fdd3149d49b4d7: Sending DeleteTablet for 3 replicas of tablet 41097523b29842fabe68c36f64c25bdf
I20260927 07:09:28.513226 11777 tablet_service.cc:1514] Processing DeleteTablet for tablet 41097523b29842fabe68c36f64c25bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:28 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:53432
I20260927 07:09:28.513310 11914 tablet_service.cc:1514] Processing DeleteTablet for tablet 41097523b29842fabe68c36f64c25bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:28 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:33144
I20260927 07:09:28.513451 12050 tablet_service.cc:1514] Processing DeleteTablet for tablet 41097523b29842fabe68c36f64c25bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:28 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:52438
I20260927 07:09:28.513522 12173 tablet_replica.cc:330] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: stopping tablet replica
I20260927 07:09:28.513667 12173 raft_consensus.cc:2229] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:28.513700 12174 tablet_replica.cc:330] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: stopping tablet replica
I20260927 07:09:28.513774 12173 raft_consensus.cc:2258] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:28.513799 12174 raft_consensus.cc:2229] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:28.513881 12174 raft_consensus.cc:2258] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:28.514457 12173 ts_tablet_manager.cc:1905] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:28.514614 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 10840
I20260927 07:09:28.516088 12174 ts_tablet_manager.cc:1905] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:28.517589 12174 ts_tablet_manager.cc:1918] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:28.517654 12174 log.cc:1196] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/41097523b29842fabe68c36f64c25bdf
I20260927 07:09:28.517887 12174 ts_tablet_manager.cc:1939] T 41097523b29842fabe68c36f64c25bdf P d64dde1fc3b94d059545eb443a4e85c9: Deleting consensus metadata
I20260927 07:09:28.518249 12173 ts_tablet_manager.cc:1918] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:28.518318 12173 log.cc:1196] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/41097523b29842fabe68c36f64c25bdf
I20260927 07:09:28.518550 12173 ts_tablet_manager.cc:1939] T 41097523b29842fabe68c36f64c25bdf P f8d46f0507454efe97b2840b0a6016ca: Deleting consensus metadata
I20260927 07:09:28.521363 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.514135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.4.174.124:45073)
0927 07:09:28.514179 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.514180 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.514189 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:28.514207 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.514208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.514217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.514271 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.515197 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.515885 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.516969 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.517228 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.517272 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.519138 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.519140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.519140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.519147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.520368 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.520671 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.520672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.520672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.520734 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.520894 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.520895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.520895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.521011 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.521109 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.521202 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.521273 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260927 07:09:28.521612 11870 connection.cc:535] server connection from 127.0.0.1:33144 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 07:09:28.522666 12172 tablet_replica.cc:330] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: stopping tablet replica
I20260927 07:09:28.522785 12172 raft_consensus.cc:2229] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:28.522900 12172 raft_consensus.cc:2258] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:28.523752 12172 ts_tablet_manager.cc:1905] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:28.525390 12172 ts_tablet_manager.cc:1918] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:28.525465 12172 log.cc:1196] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/41097523b29842fabe68c36f64c25bdf
I20260927 07:09:28.525712 12172 ts_tablet_manager.cc:1939] T 41097523b29842fabe68c36f64c25bdf P 6e2ac51ab50e41cd8d637a2cd42d19ea: Deleting consensus metadata
W20260927 07:09:28.526052 11733 connection.cc:441] server connection from 127.0.0.1:53432 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53432 (request call id 1) could send its response
I20260927 07:09:28.530488 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.523357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54742 (local address 127.4.174.124:45073)
0927 07:09:28.523389 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.523391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.523535 (+ 144us) server_negotiation.cc:407] Connection header received
0927 07:09:28.523609 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.523610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.523620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.523662 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.524367 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.525079 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.526059 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.526340 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.526379 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.528090 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.528104 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.528104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.528111 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.529452 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.529801 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.529802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.529802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.529859 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.530048 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.530049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.530049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.530152 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.530231 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.530329 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.530399 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:09:28.531522 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:28.523487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.4.174.125:37471)
0927 07:09:28.523527 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:28.523529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:28.523687 (+ 158us) server_negotiation.cc:407] Connection header received
0927 07:09:28.523770 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:28.523771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:28.523782 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:28.523810 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:28.525187 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.525926 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.526905 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:28.527121 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:28.527172 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:28.528542 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:28.528544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:28.528544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:28.528552 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:28.530442 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.530875 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.530876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.530877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.530935 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:28.531130 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:28.531130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:28.531130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:28.531216 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:28.531275 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:28.531403 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 07:09:28.531464 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:29.512033 12183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:29.511729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:33761)
0927 07:09:29.511885 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:29.511971 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260927 07:09:29.512177 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:29.511811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:59511)
0927 07:09:29.511982 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:29.512073 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":55,"threads_started":1}
W20260927 07:09:29.512259 11981 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:33097 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
I20260927 07:09:29.512355 12185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:29.511914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:34597)
0927 07:09:29.512255 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:29.512318 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":82,"threads_started":1}
W20260927 07:09:29.512365 12117 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:33097 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
W20260927 07:09:29.512614 11844 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:33097 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
I20260927 07:09:30.013783 12186 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfd23aea91a3461894fdd3149d49b4d7)
I20260927 07:09:30.013880 12187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfd23aea91a3461894fdd3149d49b4d7)
I20260927 07:09:30.013926 12186 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } }
I20260927 07:09:30.014042 12187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } }
I20260927 07:09:30.014197 12186 leader_election.cc:288] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097), 65020500fc70441b97bf3547b2ed780b (127.4.174.125:37471)
I20260927 07:09:30.014299 12187 leader_election.cc:288] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097), f8260de92b0b4045a00d10ee509503c5 (127.4.174.124:45073)
I20260927 07:09:30.014708 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.014325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54146)
0927 07:09:30.014601 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.014664 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":60,"threads_started":1}
W20260927 07:09:30.014824 11650 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
I20260927 07:09:30.014984 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.014451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54154)
0927 07:09:30.014854 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.014948 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":39,"threads_started":1}
I20260927 07:09:30.015203 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.015129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54170)
0927 07:09:30.015160 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.015180 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260927 07:09:30.015230 11456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
W20260927 07:09:30.015265 11650 leader_election.cc:334] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097): Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
I20260927 07:09:30.015618 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.015523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54178)
0927 07:09:30.015565 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.015594 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 07:09:30.015691 11456 leader_election.cc:334] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097): Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
I20260927 07:09:30.017431 12189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.014448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:45073 (local address 127.0.0.1:54752)
0927 07:09:30.014601 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.014612 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:30.014667 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:30.015001 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:30.015002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:30.015005 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:30.015072 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:30.015074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:30.015938 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:30.015939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:30.016680 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:30.016682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:30.017188 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:30.017189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:30.017240 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:30.017243 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:30.017286 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 07:09:30.017332 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":48,"threads_started":1}
I20260927 07:09:30.017428 12193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.014753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.4.174.124:45073)
0927 07:09:30.014850 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:30.014853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:30.014860 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:30.014885 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:30.014886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:30.014896 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:30.015134 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:30.015167 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:30.015868 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:30.016757 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:30.017125 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:30.017182 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:30.017193 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:30.017297 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 07:09:30.017337 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
I20260927 07:09:30.017446 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.014675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51868 (local address 127.4.174.125:37471)
0927 07:09:30.014770 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:30.014772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:30.014872 (+ 100us) server_negotiation.cc:407] Connection header received
0927 07:09:30.014965 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:30.014966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:30.014975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:30.015028 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:30.015214 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:30.015883 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:30.016770 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:30.017137 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:30.017194 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:30.017204 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:30.017319 (+ 115us) server_negotiation.cc:299] Negotiation successful
0927 07:09:30.017358 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":37,"threads_started":1}
I20260927 07:09:30.017493 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.014488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:37471 (local address 127.0.0.1:51868)
0927 07:09:30.014812 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.014822 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:30.014896 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:30.015098 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:30.015099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:30.015101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:30.015149 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:30.015150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:30.015952 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:30.015953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:30.016695 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:30.016697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:30.017201 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:30.017202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:30.017262 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:30.017265 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:30.017309 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 07:09:30.017357 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":44,"threads_started":1}
I20260927 07:09:30.017670 11668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65020500fc70441b97bf3547b2ed780b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f8260de92b0b4045a00d10ee509503c5" is_pre_election: true
I20260927 07:09:30.017781 11668 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65020500fc70441b97bf3547b2ed780b in term 1.
I20260927 07:09:30.017797 11480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8260de92b0b4045a00d10ee509503c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65020500fc70441b97bf3547b2ed780b" is_pre_election: true
I20260927 07:09:30.017897 11480 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8260de92b0b4045a00d10ee509503c5 in term 1.
I20260927 07:09:30.017938 11454 leader_election.cc:302] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65020500fc70441b97bf3547b2ed780b, f8260de92b0b4045a00d10ee509503c5; no voters: bfd23aea91a3461894fdd3149d49b4d7
I20260927 07:09:30.018015 12187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 07:09:30.018040 12187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Starting leader election (detected failure of leader bfd23aea91a3461894fdd3149d49b4d7)
I20260927 07:09:30.018059 12187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:09:30.018050 11650 leader_election.cc:302] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65020500fc70441b97bf3547b2ed780b, f8260de92b0b4045a00d10ee509503c5; no voters: bfd23aea91a3461894fdd3149d49b4d7
I20260927 07:09:30.018146 12186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 07:09:30.018185 12186 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bfd23aea91a3461894fdd3149d49b4d7)
I20260927 07:09:30.018210 12186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:09:30.018966 12187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } }
I20260927 07:09:30.018966 12186 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } }
I20260927 07:09:30.019096 12186 leader_election.cc:288] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 2 election: Requested vote from peers bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097), 65020500fc70441b97bf3547b2ed780b (127.4.174.125:37471)
I20260927 07:09:30.019109 12187 leader_election.cc:288] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [CANDIDATE]: Term 2 election: Requested vote from peers bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097), f8260de92b0b4045a00d10ee509503c5 (127.4.174.124:45073)
I20260927 07:09:30.019272 11668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65020500fc70441b97bf3547b2ed780b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f8260de92b0b4045a00d10ee509503c5"
I20260927 07:09:30.019322 12189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.019175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54184)
0927 07:09:30.019269 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.019297 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260927 07:09:30.019349 11668 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 65020500fc70441b97bf3547b2ed780b in current term 2: Already voted for candidate f8260de92b0b4045a00d10ee509503c5 in this term.
I20260927 07:09:30.019392 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.019272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54186)
0927 07:09:30.019318 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.019347 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 07:09:30.019446 11480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8260de92b0b4045a00d10ee509503c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65020500fc70441b97bf3547b2ed780b"
I20260927 07:09:30.019515 11480 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f8260de92b0b4045a00d10ee509503c5 in current term 2: Already voted for candidate 65020500fc70441b97bf3547b2ed780b in this term.
I20260927 07:09:30.019734 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.019568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54190)
0927 07:09:30.019674 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.019711 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260927 07:09:30.019752 12189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.019640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54192)
0927 07:09:30.019713 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.019734 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260927 07:09:30.019809 11650 leader_election.cc:334] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097): Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
W20260927 07:09:30.019816 11456 leader_election.cc:334] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097): Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
I20260927 07:09:30.019847 11650 leader_election.cc:302] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8260de92b0b4045a00d10ee509503c5; no voters: 65020500fc70441b97bf3547b2ed780b, bfd23aea91a3461894fdd3149d49b4d7
I20260927 07:09:30.019876 11456 leader_election.cc:302] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65020500fc70441b97bf3547b2ed780b; no voters: bfd23aea91a3461894fdd3149d49b4d7, f8260de92b0b4045a00d10ee509503c5
I20260927 07:09:30.019928 12186 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 07:09:30.019948 12187 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 07:09:30.512785 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.512538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:39077)
0927 07:09:30.512685 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.512729 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260927 07:09:30.512820 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.512606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:36531)
0927 07:09:30.512735 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.512771 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260927 07:09:30.513067 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:30.512878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:40447)
0927 07:09:30.512988 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:30.513025 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
I20260927 07:09:31.513511 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.513259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:60577)
0927 07:09:31.513404 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.513458 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
I20260927 07:09:31.513648 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.513471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:39585)
0927 07:09:31.513571 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.513601 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":47,"threads_started":1}
I20260927 07:09:31.513734 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.513259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:44133)
0927 07:09:31.513659 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.513693 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":56,"threads_started":1}
I20260927 07:09:31.598906 12202 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:31.599018 12202 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } }
I20260927 07:09:31.599177 12202 leader_election.cc:288] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097), 65020500fc70441b97bf3547b2ed780b (127.4.174.125:37471)
I20260927 07:09:31.599568 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.599360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54384)
0927 07:09:31.599477 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.599525 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260927 07:09:31.599663 11480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8260de92b0b4045a00d10ee509503c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65020500fc70441b97bf3547b2ed780b" is_pre_election: true
I20260927 07:09:31.599762 11480 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8260de92b0b4045a00d10ee509503c5 in term 2.
I20260927 07:09:31.599987 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.599893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54386)
0927 07:09:31.599926 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.599958 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260927 07:09:31.600008 11650 leader_election.cc:302] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65020500fc70441b97bf3547b2ed780b, f8260de92b0b4045a00d10ee509503c5; no voters:
W20260927 07:09:31.600075 11650 leader_election.cc:334] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097): Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
I20260927 07:09:31.600103 12202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260927 07:09:31.600152 12202 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:31.600183 12202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 2 FOLLOWER]: Advancing to term 3
I20260927 07:09:31.600872 12202 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } }
I20260927 07:09:31.600991 12202 leader_election.cc:288] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 3 election: Requested vote from peers bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097), 65020500fc70441b97bf3547b2ed780b (127.4.174.125:37471)
I20260927 07:09:31.601195 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.601109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54392)
0927 07:09:31.601141 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.601168 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260927 07:09:31.601258 11480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8260de92b0b4045a00d10ee509503c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65020500fc70441b97bf3547b2ed780b"
I20260927 07:09:31.601341 11480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 2 FOLLOWER]: Advancing to term 3
I20260927 07:09:31.601497 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.601425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54402)
0927 07:09:31.601453 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.601476 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260927 07:09:31.601562 11650 leader_election.cc:334] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097): Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
I20260927 07:09:31.601991 11480 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8260de92b0b4045a00d10ee509503c5 in term 3.
I20260927 07:09:31.602137 11650 leader_election.cc:302] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65020500fc70441b97bf3547b2ed780b, f8260de92b0b4045a00d10ee509503c5; no voters: bfd23aea91a3461894fdd3149d49b4d7
I20260927 07:09:31.602223 12202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 07:09:31.602330 12202 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [term 3 LEADER]: Becoming Leader. State: Replica: f8260de92b0b4045a00d10ee509503c5, State: Running, Role: LEADER
I20260927 07:09:31.602398 12202 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } }
I20260927 07:09:31.602751 12205 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8260de92b0b4045a00d10ee509503c5. Latest consensus state: current_term: 3 leader_uuid: "f8260de92b0b4045a00d10ee509503c5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:31.602842 12205 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:31.603029 12207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:31.603434 12207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:31.603494 12207 catalog_manager.cc:1259] Loaded cluster ID: 841d0267662c4122bf6520a898268acc
I20260927 07:09:31.603523 12207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:31.603698 12207 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:31.603765 12207 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Loaded TSK: 0
I20260927 07:09:31.603901 12207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:09:31.613478 11664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:31.615108 11664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:31.616169 11480 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [term 3 FOLLOWER]: Refusing update from remote peer f8260de92b0b4045a00d10ee509503c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260927 07:09:31.616211 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.616093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54418)
0927 07:09:31.616156 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.616183 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 07:09:31.616463 12205 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260927 07:09:31.616616 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.616514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.0.0.1:54426)
0927 07:09:31.616568 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.616593 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260927 07:09:31.616685 11650 consensus_peers.cc:487] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 -> Peer bfd23aea91a3461894fdd3149d49b4d7 (127.4.174.126:33097): Couldn't send request to peer bfd23aea91a3461894fdd3149d49b4d7. Status: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 07:09:31.617048 12209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8260de92b0b4045a00d10ee509503c5. Latest consensus state: current_term: 3 leader_uuid: "f8260de92b0b4045a00d10ee509503c5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:31.617146 12209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:09:31.617533 12205 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f8260de92b0b4045a00d10ee509503c5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:31.617628 12205 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:31.617705 12209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f8260de92b0b4045a00d10ee509503c5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bfd23aea91a3461894fdd3149d49b4d7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33097 } } peers { permanent_uuid: "65020500fc70441b97bf3547b2ed780b" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 37471 } attrs { promote: false } } peers { permanent_uuid: "f8260de92b0b4045a00d10ee509503c5" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 45073 } attrs { promote: false } } }
I20260927 07:09:31.617827 12209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65020500fc70441b97bf3547b2ed780b [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:09:31.622546 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.619366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.4.174.65:44313)
0927 07:09:31.619590 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:31.619593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:31.619601 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:31.619627 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:31.619627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:31.619638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:31.619737 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:31.619931 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.620683 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.621638 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.622222 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.622313 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.622325 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:31.622443 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 07:09:31.622477 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":71,"threads_started":1}
I20260927 07:09:31.622591 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.619286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:44313 (local address 127.0.0.1:53438)
0927 07:09:31.619316 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.619326 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:31.619379 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:31.619793 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:31.619793 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:31.619795 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:31.619847 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.619849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.620771 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.620772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.621560 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.621563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.622314 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.622315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.622380 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.622384 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:31.622434 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:09:31.622480 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":38}
I20260927 07:09:31.622785 12217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.619484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.4.174.66:43757)
0927 07:09:31.619674 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:31.619676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:31.619875 (+ 199us) server_negotiation.cc:407] Connection header received
0927 07:09:31.620014 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:31.620015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:31.620035 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:31.620096 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:31.620203 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.620666 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.621635 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.622193 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.622269 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.622281 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:31.622675 (+ 394us) server_negotiation.cc:299] Negotiation successful
0927 07:09:31.622721 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":67,"threads_started":1}
I20260927 07:09:31.623170 11777 tablet_service.cc:1467] Processing CreateTablet for tablet 3036e91a9763498c964a2d738d5830f2 (DEFAULT_TABLE table=test-table [id=b0352cd41caf49599302c8c5e11a2b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:31.622592 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.619308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43757 (local address 127.0.0.1:33152)
0927 07:09:31.619453 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.619735 (+ 282us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:31.619893 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:31.620108 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:31.620109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:31.620110 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:31.620155 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.620157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.620731 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.620732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.621558 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.621560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.622312 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.622313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.622380 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.622386 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 07:09:31.622436 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:09:31.622480 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":29,"thread_start_us":76,"threads_started":1}
I20260927 07:09:31.623313 11777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3036e91a9763498c964a2d738d5830f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:31.623440 11914 tablet_service.cc:1467] Processing CreateTablet for tablet 3036e91a9763498c964a2d738d5830f2 (DEFAULT_TABLE table=test-table [id=b0352cd41caf49599302c8c5e11a2b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:31.623557 11914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3036e91a9763498c964a2d738d5830f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:31.623677 12218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.619501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52464 (local address 127.4.174.67:41547)
0927 07:09:31.619605 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:31.619607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:31.619684 (+ 77us) server_negotiation.cc:407] Connection header received
0927 07:09:31.619734 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:31.619735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:31.619748 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:31.619802 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:31.619931 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.620687 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.622702 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.623275 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.623369 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.623383 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:31.623563 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 07:09:31.623611 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":62,"threads_started":1}
I20260927 07:09:31.623849 12214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.619363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:41547 (local address 127.0.0.1:52464)
0927 07:09:31.619605 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.619613 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:31.619682 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:31.619803 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:31.619803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:31.619805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:31.619848 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.619849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.620771 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.620771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.622633 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.622636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.623412 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.623413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.623467 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.623471 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:31.623743 (+ 272us) client_negotiation.cc:240] Negotiation successful
0927 07:09:31.623784 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":25,"thread_start_us":50,"threads_started":1}
I20260927 07:09:31.624089 12050 tablet_service.cc:1467] Processing CreateTablet for tablet 3036e91a9763498c964a2d738d5830f2 (DEFAULT_TABLE table=test-table [id=b0352cd41caf49599302c8c5e11a2b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:31.624186 12050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3036e91a9763498c964a2d738d5830f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:31.624593 12219 tablet_bootstrap.cc:492] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea: Bootstrap starting.
I20260927 07:09:31.624907 12219 tablet_bootstrap.cc:654] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:31.625519 12220 tablet_bootstrap.cc:492] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: Bootstrap starting.
I20260927 07:09:31.625525 12222 tablet_bootstrap.cc:492] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: Bootstrap starting.
I20260927 07:09:31.625617 12219 tablet_bootstrap.cc:492] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea: No bootstrap required, opened a new log
I20260927 07:09:31.625689 12219 ts_tablet_manager.cc:1397] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:31.625829 12219 raft_consensus.cc:348] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.625903 12219 raft_consensus.cc:374] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:31.625903 12222 tablet_bootstrap.cc:654] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:31.625895 12220 tablet_bootstrap.cc:654] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:31.625924 12219 raft_consensus.cc:729] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2ac51ab50e41cd8d637a2cd42d19ea, State: Initialized, Role: FOLLOWER
I20260927 07:09:31.626001 12219 consensus_queue.cc:260] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.626185 12219 ts_tablet_manager.cc:1428] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:31.626431 12222 tablet_bootstrap.cc:492] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: No bootstrap required, opened a new log
I20260927 07:09:31.626441 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.626315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:47837)
0927 07:09:31.626382 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.626414 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 07:09:31.626485 12222 ts_tablet_manager.cc:1397] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:31.626616 12222 raft_consensus.cc:348] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.626682 12222 raft_consensus.cc:374] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:31.626700 12222 raft_consensus.cc:729] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64dde1fc3b94d059545eb443a4e85c9, State: Initialized, Role: FOLLOWER
I20260927 07:09:31.626729 11843 heartbeater.cc:503] Master 127.4.174.124:45073 was elected leader, sending a full tablet report...
I20260927 07:09:31.626740 12222 consensus_queue.cc:260] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.626900 12222 ts_tablet_manager.cc:1428] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:31.627070 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.626939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:59625)
0927 07:09:31.627016 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.627046 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260927 07:09:31.627391 12220 tablet_bootstrap.cc:492] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: No bootstrap required, opened a new log
I20260927 07:09:31.627413 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.627276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:60485)
0927 07:09:31.627364 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.627387 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 07:09:31.627506 12220 ts_tablet_manager.cc:1397] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:31.627679 12220 raft_consensus.cc:348] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.627799 12220 raft_consensus.cc:374] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:31.627858 12220 raft_consensus.cc:729] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8d46f0507454efe97b2840b0a6016ca, State: Initialized, Role: FOLLOWER
I20260927 07:09:31.627872 12116 heartbeater.cc:503] Master 127.4.174.124:45073 was elected leader, sending a full tablet report...
I20260927 07:09:31.627928 12220 consensus_queue.cc:260] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.628036 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.627924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:51115)
0927 07:09:31.627988 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.628011 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260927 07:09:31.628140 12220 ts_tablet_manager.cc:1428] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 07:09:31.628453 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.628363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:35661)
0927 07:09:31.628398 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.628426 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 07:09:31.628719 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.628639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:55925)
0927 07:09:31.628669 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.628692 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260927 07:09:31.686089 12225 raft_consensus.cc:482] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:31.686195 12225 raft_consensus.cc:504] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.686524 12225 leader_election.cc:288] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757), d64dde1fc3b94d059545eb443a4e85c9 (127.4.174.67:41547)
I20260927 07:09:31.689389 12218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.686803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:41137 (local address 127.4.174.67:41547)
0927 07:09:31.686851 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:31.686853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:31.686864 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:09:31.686950 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:31.686951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:31.686959 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:31.687005 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:31.687170 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.687713 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.688719 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.689081 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.689142 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.689153 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:31.689277 (+ 124us) server_negotiation.cc:299] Negotiation successful
0927 07:09:31.689313 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:31.689400 12217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.686869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:43305 (local address 127.4.174.66:43757)
0927 07:09:31.686920 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:31.686922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:31.687095 (+ 173us) server_negotiation.cc:407] Connection header received
0927 07:09:31.687129 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:31.687130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:31.687140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:31.687188 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:31.687306 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.687746 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.688733 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.689081 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.689142 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.689151 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:31.689303 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 07:09:31.689335 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 07:09:31.689409 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.686739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:41547 (local address 127.4.174.65:41137)
0927 07:09:31.686784 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.686796 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:31.686853 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:31.687012 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:31.687013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:31.687015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:31.687093 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.687095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.687787 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.687788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.688639 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.688642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.689150 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.689151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.689217 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.689220 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:31.689259 (+ 39us) client_negotiation.cc:240] Negotiation successful
0927 07:09:31.689299 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":30}
I20260927 07:09:31.689437 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.686747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43757 (local address 127.4.174.65:43305)
0927 07:09:31.687028 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.687037 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:31.687093 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:31.687191 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:31.687192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:31.687194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:31.687250 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.687252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.687818 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.687819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.688648 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:31.688650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.689173 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.689175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:31.689236 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.689240 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:31.689292 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 07:09:31.689331 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":97,"thread_start_us":127,"threads_started":1}
I20260927 07:09:31.689677 11934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3036e91a9763498c964a2d738d5830f2" candidate_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d46f0507454efe97b2840b0a6016ca" is_pre_election: true
I20260927 07:09:31.689771 11934 raft_consensus.cc:2454] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e2ac51ab50e41cd8d637a2cd42d19ea in term 0.
I20260927 07:09:31.689721 12070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3036e91a9763498c964a2d738d5830f2" candidate_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64dde1fc3b94d059545eb443a4e85c9" is_pre_election: true
I20260927 07:09:31.689852 12070 raft_consensus.cc:2454] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e2ac51ab50e41cd8d637a2cd42d19ea in term 0.
I20260927 07:09:31.689930 11730 leader_election.cc:302] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2ac51ab50e41cd8d637a2cd42d19ea, f8d46f0507454efe97b2840b0a6016ca; no voters:
I20260927 07:09:31.690052 12225 raft_consensus.cc:2781] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:31.690089 12225 raft_consensus.cc:482] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:31.690114 12225 raft_consensus.cc:3037] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:31.690937 12225 raft_consensus.cc:504] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.691069 12225 leader_election.cc:288] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [CANDIDATE]: Term 1 election: Requested vote from peers f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757), d64dde1fc3b94d059545eb443a4e85c9 (127.4.174.67:41547)
I20260927 07:09:31.691223 12070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3036e91a9763498c964a2d738d5830f2" candidate_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64dde1fc3b94d059545eb443a4e85c9"
I20260927 07:09:31.691246 11934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3036e91a9763498c964a2d738d5830f2" candidate_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d46f0507454efe97b2840b0a6016ca"
I20260927 07:09:31.691293 12070 raft_consensus.cc:3037] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:31.691309 11934 raft_consensus.cc:3037] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:31.691895 11934 raft_consensus.cc:2454] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2ac51ab50e41cd8d637a2cd42d19ea in term 1.
I20260927 07:09:31.692030 12070 raft_consensus.cc:2454] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2ac51ab50e41cd8d637a2cd42d19ea in term 1.
I20260927 07:09:31.692046 11730 leader_election.cc:302] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2ac51ab50e41cd8d637a2cd42d19ea, f8d46f0507454efe97b2840b0a6016ca; no voters:
I20260927 07:09:31.692119 12225 raft_consensus.cc:2781] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:31.692287 12225 raft_consensus.cc:686] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 LEADER]: Becoming Leader. State: Replica: 6e2ac51ab50e41cd8d637a2cd42d19ea, State: Running, Role: LEADER
I20260927 07:09:31.692446 12225 consensus_queue.cc:237] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } }
I20260927 07:09:31.692693 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.692507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:43037)
0927 07:09:31.692630 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.692665 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260927 07:09:31.693143 11664 catalog_manager.cc:5495] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e2ac51ab50e41cd8d637a2cd42d19ea (127.4.174.65). New cstate: current_term: 1 leader_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e2ac51ab50e41cd8d637a2cd42d19ea" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:31.715631 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.713242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53450 (local address 127.4.174.65:44313)
0927 07:09:31.713289 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:31.713291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:31.713299 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:31.713336 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:31.713336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:31.713355 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:31.713420 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:31.713566 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.714029 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.714850 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:31.715079 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:31.715138 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:31.715337 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:31.715451 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:31.715511 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:31.715542 (+ 31us) server_negotiation.cc:299] Negotiation successful
0927 07:09:31.715562 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 07:09:31.716694 11934 raft_consensus.cc:1261] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 1 FOLLOWER]: Refusing update from remote peer 6e2ac51ab50e41cd8d637a2cd42d19ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:31.716806 12070 raft_consensus.cc:1261] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 1 FOLLOWER]: Refusing update from remote peer 6e2ac51ab50e41cd8d637a2cd42d19ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:31.717088 12225 consensus_queue.cc:1035] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8d46f0507454efe97b2840b0a6016ca" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:31.717231 12229 consensus_queue.cc:1035] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "d64dde1fc3b94d059545eb443a4e85c9" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 41547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:31.717634 11980 heartbeater.cc:503] Master 127.4.174.124:45073 was elected leader, sending a full tablet report...
I20260927 07:09:31.717859 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.717746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:33151)
0927 07:09:31.717801 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.717831 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 07:09:31.718369 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.718252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:49575)
0927 07:09:31.718302 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.718337 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 07:09:31.718886 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.718788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.67:32773)
0927 07:09:31.718821 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.718845 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260927 07:09:31.719030 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.718928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.65:56199)
0927 07:09:31.718963 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.718986 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260927 07:09:31.719834 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.719737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33097 (local address 127.4.174.66:45113)
0927 07:09:31.719779 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.719805 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 07:09:31.721011 11664 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:31.721101 11664 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:31.722249 11664 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8260de92b0b4045a00d10ee509503c5: Sending DeleteTablet for 3 replicas of tablet 3036e91a9763498c964a2d738d5830f2
I20260927 07:09:31.722584 11914 tablet_service.cc:1514] Processing DeleteTablet for tablet 3036e91a9763498c964a2d738d5830f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:31 UTC) from {username='oryx', principal='oryx/127.4.174.124@KRBTEST.COM'} at 127.0.0.1:33152
I20260927 07:09:31.722591 11777 tablet_service.cc:1514] Processing DeleteTablet for tablet 3036e91a9763498c964a2d738d5830f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:31 UTC) from {username='oryx', principal='oryx/127.4.174.124@KRBTEST.COM'} at 127.0.0.1:53438
I20260927 07:09:31.722595 12050 tablet_service.cc:1514] Processing DeleteTablet for tablet 3036e91a9763498c964a2d738d5830f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:31 UTC) from {username='oryx', principal='oryx/127.4.174.124@KRBTEST.COM'} at 127.0.0.1:52464
I20260927 07:09:31.722903 12245 tablet_replica.cc:330] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea: stopping tablet replica
I20260927 07:09:31.722961 12245 raft_consensus.cc:2229] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:31.723184 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 11618
I20260927 07:09:31.723096 12245 raft_consensus.cc:2258] T 3036e91a9763498c964a2d738d5830f2 P 6e2ac51ab50e41cd8d637a2cd42d19ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:31.723444 12244 tablet_replica.cc:330] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: stopping tablet replica
I20260927 07:09:31.723507 12244 raft_consensus.cc:2229] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:31.723544 12244 raft_consensus.cc:2258] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:31.723769 12243 tablet_replica.cc:330] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: stopping tablet replica
I20260927 07:09:31.723901 12243 raft_consensus.cc:2229] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:31.723928 12243 raft_consensus.cc:2258] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:31.723958 12244 ts_tablet_manager.cc:1905] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:31.724160 12243 ts_tablet_manager.cc:1905] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:31.725523 12244 ts_tablet_manager.cc:1918] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:31.725554 12243 ts_tablet_manager.cc:1918] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:31.725572 12244 log.cc:1196] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/3036e91a9763498c964a2d738d5830f2
I20260927 07:09:31.725595 12243 log.cc:1196] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/3036e91a9763498c964a2d738d5830f2
I20260927 07:09:31.725775 12243 ts_tablet_manager.cc:1939] T 3036e91a9763498c964a2d738d5830f2 P f8d46f0507454efe97b2840b0a6016ca: Deleting consensus metadata
I20260927 07:09:31.725826 12244 ts_tablet_manager.cc:1939] T 3036e91a9763498c964a2d738d5830f2 P d64dde1fc3b94d059545eb443a4e85c9: Deleting consensus metadata
I20260927 07:09:31.726105 11648 catalog_manager.cc:4841] TS f8d46f0507454efe97b2840b0a6016ca (127.4.174.66:43757): tablet 3036e91a9763498c964a2d738d5830f2 (table test-table [id=b0352cd41caf49599302c8c5e11a2b29]) successfully deleted
I20260927 07:09:31.726250 11649 catalog_manager.cc:4841] TS d64dde1fc3b94d059545eb443a4e85c9 (127.4.174.67:41547): tablet 3036e91a9763498c964a2d738d5830f2 (table test-table [id=b0352cd41caf49599302c8c5e11a2b29]) successfully deleted
I20260927 07:09:31.728639 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:31.728520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:44313 (local address 127.0.0.1:53456)
0927 07:09:31.728581 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:31.728609 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:44313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 07:09:31.728787 11649 catalog_manager.cc:4584] TS 6e2ac51ab50e41cd8d637a2cd42d19ea (127.4.174.65:44313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3036e91a9763498c964a2d738d5830f2: Network error: Client connection negotiation failed: client connection to 127.4.174.65:44313: connect: Connection refused (error 111)
I20260927 07:09:31.729543 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 11851
I20260927 07:09:31.735775 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 11987
I20260927 07:09:31.740450 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 11336
I20260927 07:09:31.744398 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 11524
2026-09-27T07:09:31Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6102 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:31 dist-test-slave-zw7p krb5kdc[12248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:31 dist-test-slave-zw7p krb5kdc[12248](info): set up 2 sockets
Sep 27 07:09:31 dist-test-slave-zw7p krb5kdc[12248](info): commencing operation
krb5kdc: starting...
W20260927 07:09:32.698741 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:32 dist-test-slave-zw7p krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.126.keytab.
Entry for principal oryx/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.126.keytab.
I20260927 07:09:32.809239 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:41397
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.126.keytab
--principal=oryx/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:41397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:32.888121 12261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:32.888324 12261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:32.888345 12261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:32.889521 12261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:32.889572 12261 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 07:09:32.889585 12261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:32.889595 12261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:32.889607 12261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:32.889618 12261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:32.891188 12261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:41397
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.126.keytab
--principal=oryx/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:41397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:32.891436 12261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:32.891644 12261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:32.896211 12266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:32.896247 12269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:32.896234 12261 server_base.cc:1034] running on GCE node
W20260927 07:09:32.896420 12267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:32.896659 12261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:32.896904 12261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:32.898061 12261 hybrid_clock.cc:648] HybridClock initialized: now 1790492972898051 us; error 33 us; skew 500 ppm
Sep 27 07:09:32 dist-test-slave-zw7p krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492972, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:32.900573 12261 init.cc:377] Logged in from keytab as oryx/127.4.174.126@KRBTEST.COM (short username oryx)
I20260927 07:09:32.901008 12261 webserver.cc:460] Webserver started at http://127.4.174.126:33263/ using document root <none> and password file <none>
I20260927 07:09:32.901221 12261 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:32.901266 12261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:32.901408 12261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:32.902441 12261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "84d45cb3241c42f584ea251fe5398782"
format_stamp: "Formatted at 2026-09-27 07:09:32 on dist-test-slave-zw7p"
server_key: "869c773c13aa5a6df1464e336a5eae81"
server_key_iv: "5870a2881bc15dea2f47cd6986f9e2e9"
server_key_version: "encryptionkey@0"
I20260927 07:09:32.902814 12261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "84d45cb3241c42f584ea251fe5398782"
format_stamp: "Formatted at 2026-09-27 07:09:32 on dist-test-slave-zw7p"
server_key: "869c773c13aa5a6df1464e336a5eae81"
server_key_iv: "5870a2881bc15dea2f47cd6986f9e2e9"
server_key_version: "encryptionkey@0"
I20260927 07:09:32.904138 12261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:32.904927 12276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:32.905097 12261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:32.905176 12261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "84d45cb3241c42f584ea251fe5398782"
format_stamp: "Formatted at 2026-09-27 07:09:32 on dist-test-slave-zw7p"
server_key: "869c773c13aa5a6df1464e336a5eae81"
server_key_iv: "5870a2881bc15dea2f47cd6986f9e2e9"
server_key_version: "encryptionkey@0"
I20260927 07:09:32.905251 12261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:32.917547 12261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:32.920076 12261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:32.920279 12261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:32.926017 12261 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:41397
I20260927 07:09:32.926072 12338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:41397 every 8 connection(s)
I20260927 07:09:32.926426 12261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:32.927160 12339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:32.929406 12339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: Bootstrap starting.
I20260927 07:09:32.929963 12339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:32.930225 12339 log.cc:824] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:32.930897 12339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: No bootstrap required, opened a new log
I20260927 07:09:32.932848 12339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d45cb3241c42f584ea251fe5398782" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41397 } }
I20260927 07:09:32.932984 12339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:32.933015 12339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d45cb3241c42f584ea251fe5398782, State: Initialized, Role: FOLLOWER
I20260927 07:09:32.933151 12339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d45cb3241c42f584ea251fe5398782" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41397 } }
I20260927 07:09:32.933211 12339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:32.933239 12339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:32.933283 12339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:32.933511 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 12261
I20260927 07:09:32.933605 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:32.933799 4793 external_mini_cluster.cc:1428] Setting key acb65d1639807047db6c6419407484ab
I20260927 07:09:32.933811 12339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d45cb3241c42f584ea251fe5398782" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41397 } }
I20260927 07:09:32.933910 12339 leader_election.cc:302] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84d45cb3241c42f584ea251fe5398782; no voters:
I20260927 07:09:32.934069 12339 leader_election.cc:288] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:32.934405 12339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:32.934094 12342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:32.934689 12342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [term 1 LEADER]: Becoming Leader. State: Replica: 84d45cb3241c42f584ea251fe5398782, State: Running, Role: LEADER
I20260927 07:09:32.934811 12342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d45cb3241c42f584ea251fe5398782" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41397 } }
I20260927 07:09:32.935166 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84d45cb3241c42f584ea251fe5398782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d45cb3241c42f584ea251fe5398782" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41397 } } }
I20260927 07:09:32.935238 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [sys.catalog]: This master's current role is: LEADER
W20260927 07:09:32.936126 12360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:09:32.936182 12360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:09:32.937090 12346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84d45cb3241c42f584ea251fe5398782. Latest consensus state: current_term: 1 leader_uuid: "84d45cb3241c42f584ea251fe5398782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d45cb3241c42f584ea251fe5398782" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41397 } } }
I20260927 07:09:32.937166 12346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:32.937263 12353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:32.937417 12353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:32.939277 12353 catalog_manager.cc:1347] Generated new cluster ID: 98179a5a13354cbb9cbf1364123439ef
I20260927 07:09:32.939323 12353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:09:32 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:32.944685 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:32.934255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48840 (local address 127.4.174.126:41397)
0927 07:09:32.934455 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:32.934464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:32.934499 (+ 35us) server_negotiation.cc:407] Connection header received
0927 07:09:32.934643 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:32.934652 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:32.934710 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:32.934848 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:32.935597 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:32.936761 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:32.937758 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:32.938248 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:32.938331 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:32.941000 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:32.941004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:32.941004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:32.941015 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:32.942869 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:32.943224 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:32.943227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:32.943227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:32.943320 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:32.943486 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:32.943487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:32.943487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:32.943754 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:32.943889 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:32.944328 (+ 439us) server_negotiation.cc:299] Negotiation successful
0927 07:09:32.944470 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260927 07:09:32.948886 12353 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:32.949375 12353 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:32.965497 12353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: Generated new TSK 0
I20260927 07:09:32.965694 12353 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab.
Entry for principal oryx/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab.
I20260927 07:09:32.997680 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41397
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab
--principal=oryx/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:33.077282 12365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:33.077495 12365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:33.077520 12365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:33.078707 12365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:33.078749 12365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:33.078768 12365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:33.078784 12365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:33.080272 12365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab
--principal=oryx/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:41397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:33.080488 12365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:33.080681 12365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:33.085664 12371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:33.085690 12370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:33.085855 12365 server_base.cc:1034] running on GCE node
W20260927 07:09:33.085758 12373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:33.086126 12365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:33.086360 12365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:33.087512 12365 hybrid_clock.cc:648] HybridClock initialized: now 1790492973087482 us; error 46 us; skew 500 ppm
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:33.089972 12365 init.cc:377] Logged in from keytab as oryx/127.4.174.65@KRBTEST.COM (short username oryx)
I20260927 07:09:33.090376 12365 webserver.cc:460] Webserver started at http://127.4.174.65:45551/ using document root <none> and password file <none>
I20260927 07:09:33.090622 12365 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:33.090669 12365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:33.090791 12365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:33.091799 12365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "b75992d841c246f18ca0753f9c571d9b"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "58eab5f5eec3bbf8b8c9c8a04d5e6ddf"
server_key_iv: "bd57c4f6182801df43122bf06e917787"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.092113 12365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "b75992d841c246f18ca0753f9c571d9b"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "58eab5f5eec3bbf8b8c9c8a04d5e6ddf"
server_key_iv: "bd57c4f6182801df43122bf06e917787"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.093461 12365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:33.094133 12380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.094302 12365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:33.094384 12365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "b75992d841c246f18ca0753f9c571d9b"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "58eab5f5eec3bbf8b8c9c8a04d5e6ddf"
server_key_iv: "bd57c4f6182801df43122bf06e917787"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.094448 12365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:33.117897 12365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:33.120380 12365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:33.120543 12365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:33.121151 12365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:33.121201 12365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.121230 12365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:33.121281 12365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:33.127770 12365 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:42435
I20260927 07:09:33.127980 12495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:42435 every 8 connection(s)
I20260927 07:09:33.128175 12365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:33.131448 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.121555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:53121 (local address 127.4.174.126:41397)
0927 07:09:33.121604 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.121607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.121839 (+ 232us) server_negotiation.cc:407] Connection header received
0927 07:09:33.122365 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.122367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.122379 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.122429 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.123281 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.124052 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.125067 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.125402 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.125472 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.128256 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.128258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.128258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.128270 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.129708 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.130142 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.130143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.130143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.130192 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.130718 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.130720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.130720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.130867 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.130939 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.131271 (+ 332us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.131379 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:33.131510 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.121555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41397 (local address 127.4.174.65:53121)
0927 07:09:33.121700 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.121746 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.122272 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.122513 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.122517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.122878 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:33.123158 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.123166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.124135 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.124136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.124980 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.124983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.125551 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.125552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.125718 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.126338 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:33.126350 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:33.128151 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:33.129780 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.129787 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.129789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.130060 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.130233 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.130234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.130234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.130612 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.130953 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:33.130958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:33.131108 (+ 150us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.131191 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.131339 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260927 07:09:33.132015 12496 heartbeater.cc:347] Connected to a master server at 127.4.174.126:41397
I20260927 07:09:33.132107 12496 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:33.132308 12496 heartbeater.cc:511] Master 127.4.174.126:41397 requested a full tablet report, sending...
I20260927 07:09:33.132387 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 12365
I20260927 07:09:33.132493 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:33.132719 4793 external_mini_cluster.cc:1428] Setting key 72c09fdfc4e991d292e3e28a677447f5
I20260927 07:09:33.132886 12292 ts_manager.cc:194] Registered new tserver with Master: b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435)
I20260927 07:09:33.133718 12292 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:53121
I20260927 07:09:33.140285 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.133071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41397 (local address 127.4.174.65:52101)
0927 07:09:33.133942 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.133954 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.134014 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.134138 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.134138 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.134333 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:33.134405 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.134406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.135279 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.135280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.135894 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.135897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.136353 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.136354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.136425 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.136931 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:33.136936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:33.137255 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:33.139288 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.139289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.139289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.139456 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.139678 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.139678 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.139679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.139730 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.140015 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:33.140015 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:33.140053 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.140119 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.140209 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":70,"threads_started":1}
I20260927 07:09:33.140336 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.133070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:52101 (local address 127.4.174.126:41397)
0927 07:09:33.133108 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.133110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.134001 (+ 891us) server_negotiation.cc:407] Connection header received
0927 07:09:33.134070 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.134070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.134082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.134130 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.134473 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.135197 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.135967 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.136240 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.136298 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.137347 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.137349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.137349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.137357 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.138814 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.139535 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.139536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.139536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.139612 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.139780 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.139781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.139782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.139932 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.139992 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.140152 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.140238 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for oryx/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab.
Entry for principal oryx/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab.
I20260927 07:09:33.157779 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41397
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab
--principal=oryx/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:33.236913 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:33.237114 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:33.237138 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:33.238237 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:33.238269 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:33.238289 12505 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:33.238305 12505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:33.239768 12505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab
--principal=oryx/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:41397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:33.239964 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:33.240131 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:33.244706 12513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:33.244880 12505 server_base.cc:1034] running on GCE node
W20260927 07:09:33.244720 12510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:33.244863 12511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:33.245211 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:33.245440 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:33.246599 12505 hybrid_clock.cc:648] HybridClock initialized: now 1790492973246585 us; error 29 us; skew 500 ppm
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:33.249055 12505 init.cc:377] Logged in from keytab as oryx/127.4.174.66@KRBTEST.COM (short username oryx)
I20260927 07:09:33.249500 12505 webserver.cc:460] Webserver started at http://127.4.174.66:38353/ using document root <none> and password file <none>
I20260927 07:09:33.249701 12505 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:33.249748 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:33.249881 12505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:33.250975 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "1edc62022738427ea3884a4b90f606d1"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "1b3ce308e4acd450efdf52251eaa70b7"
server_key_iv: "689d098ae044c547bfaa53a07244de54"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.251310 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "1edc62022738427ea3884a4b90f606d1"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "1b3ce308e4acd450efdf52251eaa70b7"
server_key_iv: "689d098ae044c547bfaa53a07244de54"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.252609 12505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:33.253293 12520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.253440 12505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:33.253521 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "1edc62022738427ea3884a4b90f606d1"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "1b3ce308e4acd450efdf52251eaa70b7"
server_key_iv: "689d098ae044c547bfaa53a07244de54"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.253585 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:33.264227 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:33.265892 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:33.266013 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:33.266495 12505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:33.266533 12505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.266619 12505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:33.266646 12505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:33.273123 12505 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:36533
I20260927 07:09:33.273458 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:33.273809 12635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:36533 every 8 connection(s)
I20260927 07:09:33.275694 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.266846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:51979 (local address 127.4.174.126:41397)
0927 07:09:33.266916 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.266919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.267112 (+ 193us) server_negotiation.cc:407] Connection header received
0927 07:09:33.267779 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.267780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.267792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.267862 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.268682 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.269248 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.270230 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.270421 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.270476 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.273112 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.273114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.273114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.273121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.274462 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.274844 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.274845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.274846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.274908 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.275113 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.275114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.275114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.275227 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.275282 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.275531 (+ 249us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.275606 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 07:09:33.275858 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.266833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41397 (local address 127.4.174.66:51979)
0927 07:09:33.266979 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.267025 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.267658 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.267942 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.267947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.268290 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:33.268592 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.268598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.269332 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.269333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.270150 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.270153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.270491 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.270491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.270649 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.271085 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:33.271093 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:33.273029 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:33.274531 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.274536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.274538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.274781 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.274959 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.274960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.274960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.275061 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.275278 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:33.275283 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:33.275419 (+ 136us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.275504 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.275660 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260927 07:09:33.276333 12636 heartbeater.cc:347] Connected to a master server at 127.4.174.126:41397
I20260927 07:09:33.276432 12636 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:33.276614 12636 heartbeater.cc:511] Master 127.4.174.126:41397 requested a full tablet report, sending...
I20260927 07:09:33.276994 12292 ts_manager.cc:194] Registered new tserver with Master: 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533)
I20260927 07:09:33.277485 12292 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:51979
I20260927 07:09:33.282055 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.277325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:56799 (local address 127.4.174.126:41397)
0927 07:09:33.277359 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.277361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.277475 (+ 114us) server_negotiation.cc:407] Connection header received
0927 07:09:33.277528 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.277529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.277539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.277581 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.277914 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.278370 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.279075 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.279267 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.279318 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.280062 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.280063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.280064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.280070 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.281107 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.281380 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.281381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.281381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.281429 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.281581 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.281581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.281582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.281713 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.281767 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.281905 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.281964 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:09:33.282068 12642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.277276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41397 (local address 127.4.174.66:56799)
0927 07:09:33.277418 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.277427 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.277478 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.277644 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.277645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.277786 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:33.277845 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.277847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.278448 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.278449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.279008 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.279010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.279320 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.279321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.279381 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.279734 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:33.279738 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:33.279992 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:33.281173 (+ 1181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.281174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.281174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.281324 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.281479 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.281480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.281480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.281533 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.281767 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:33.281768 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:33.281812 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.281871 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.281964 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260927 07:09:33.282452 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 12505
I20260927 07:09:33.282578 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:33.282761 4793 external_mini_cluster.cc:1428] Setting key 3116c922ce86fe7ac5f5780f34805a9d
WARNING: no policy specified for oryx/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab.
Entry for principal oryx/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab.
I20260927 07:09:33.305576 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41397
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab
--principal=oryx/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:33.384145 12645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:33.384348 12645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:33.384380 12645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:33.385545 12645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:33.385587 12645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:33.385615 12645 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:09:33.385638 12645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:33.387140 12645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab
--principal=oryx/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.174.126:41397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:33.387365 12645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:33.387573 12645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:33.392161 12653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:33.392252 12651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:33.392247 12650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:33.392675 12645 server_base.cc:1034] running on GCE node
I20260927 07:09:33.392848 12645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:33.393064 12645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:33.394217 12645 hybrid_clock.cc:648] HybridClock initialized: now 1790492973394206 us; error 31 us; skew 500 ppm
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:33.396564 12645 init.cc:377] Logged in from keytab as oryx/127.4.174.67@KRBTEST.COM (short username oryx)
I20260927 07:09:33.397003 12645 webserver.cc:460] Webserver started at http://127.4.174.67:33623/ using document root <none> and password file <none>
I20260927 07:09:33.397181 12645 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:33.397223 12645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:33.397341 12645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:33.398342 12645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "0c6b951e028141a38b91443f73cd8aaf"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "0d2ae9229ee01c6e6607effb955bc4df"
server_key_iv: "bf6bca08a664d54852cd757204a39089"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.398681 12645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "0c6b951e028141a38b91443f73cd8aaf"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "0d2ae9229ee01c6e6607effb955bc4df"
server_key_iv: "bf6bca08a664d54852cd757204a39089"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.399895 12645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:33.400539 12660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.400717 12645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.400805 12645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "0c6b951e028141a38b91443f73cd8aaf"
format_stamp: "Formatted at 2026-09-27 07:09:33 on dist-test-slave-zw7p"
server_key: "0d2ae9229ee01c6e6607effb955bc4df"
server_key_iv: "bf6bca08a664d54852cd757204a39089"
server_key_version: "encryptionkey@0"
I20260927 07:09:33.400871 12645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:33.421370 12645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:33.423463 12645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:33.423580 12645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:33.424100 12645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:33.424139 12645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.424165 12645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:33.424180 12645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:33.430459 12645 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:39549
I20260927 07:09:33.430542 12775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:39549 every 8 connection(s)
I20260927 07:09:33.430910 12645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:33.433176 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.424660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:42137 (local address 127.4.174.126:41397)
0927 07:09:33.424710 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.424712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.424927 (+ 215us) server_negotiation.cc:407] Connection header received
0927 07:09:33.425424 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.425425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.425440 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.425512 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.426302 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.427092 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.428088 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.428390 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.428457 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.430764 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.430766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.430766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.430774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.432070 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.432452 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.432453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.432454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.432517 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.432715 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.432715 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.432716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.432817 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.432893 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.433053 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.433113 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 07:09:33.433399 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.424649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41397 (local address 127.4.174.67:42137)
0927 07:09:33.424794 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.424830 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.425316 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.425585 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.425589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.425922 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:33.426202 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.426208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.427186 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.427187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.428001 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.428003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.428469 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.428470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.428621 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.429088 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:33.429096 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:33.430683 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:33.432158 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.432164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.432167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.432385 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.432580 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.432581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.432581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.432672 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.432851 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:33.432854 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:33.432965 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.433037 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.433189 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260927 07:09:33.433810 12776 heartbeater.cc:347] Connected to a master server at 127.4.174.126:41397
I20260927 07:09:33.433890 12776 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:33.434082 12776 heartbeater.cc:511] Master 127.4.174.126:41397 requested a full tablet report, sending...
I20260927 07:09:33.434499 12293 ts_manager.cc:194] Registered new tserver with Master: 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549)
I20260927 07:09:33.434962 12293 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:42137
I20260927 07:09:33.439769 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.434784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:38145 (local address 127.4.174.126:41397)
0927 07:09:33.434825 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.434826 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.434905 (+ 79us) server_negotiation.cc:407] Connection header received
0927 07:09:33.434950 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.434952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.434961 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.435003 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.435360 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.435997 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.436662 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.436867 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.436918 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.437729 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.437731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.437731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.437739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.438828 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.439108 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.439109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.439109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.439159 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.439311 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.439312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.439312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.439419 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.439471 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.439612 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.439682 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:09:33.439787 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.434723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41397 (local address 127.4.174.67:38145)
0927 07:09:33.434838 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.434846 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.434899 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.435005 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.435005 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.435208 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:33.435268 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.435270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.436082 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.436083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.436595 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.436596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.436921 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.436922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.436984 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.437336 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:33.437341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:33.437630 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:33.438895 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.438896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.438896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.439050 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.439208 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.439209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.439210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.439264 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.439470 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:33.439471 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:33.439514 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.439573 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.439681 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
I20260927 07:09:33.440352 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 12645
I20260927 07:09:33.440457 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:33.440670 4793 external_mini_cluster.cc:1428] Setting key 2700c308b4ca36444c2dc5d1bf71eef5
I20260927 07:09:33.441618 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:09:33.461274 12345 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:33.456176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.4.174.126:41397)
0927 07:09:33.456226 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.456228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.456239 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:09:33.456307 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.456308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.456316 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.456356 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.456894 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.457437 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.458136 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.458315 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.458368 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.461194 (+ 2826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48848: BlockingRecv error: recv got EOF from 127.0.0.1:48848 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:33.467851 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.461961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48852 (local address 127.4.174.126:41397)
0927 07:09:33.462017 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.462019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.462095 (+ 76us) server_negotiation.cc:407] Connection header received
0927 07:09:33.462153 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.462153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.462160 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.462203 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.462617 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.463062 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.463665 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.463833 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.463871 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.465816 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.465818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.465818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.465825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.466922 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.467203 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.467204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.467204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.467252 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.467408 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.467409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.467409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.467514 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.467595 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.467695 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.467763 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 07:09:33.469368 12293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:33.470974 12293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:33.477247 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.473148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36533 (local address 127.0.0.1:53912)
0927 07:09:33.473389 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.473398 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.473439 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.473634 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.473637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.473639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:33.473715 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.473717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.475032 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.475033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.476294 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.476296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.476954 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.476955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.477037 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.477052 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.477116 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.477169 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":81,"threads_started":1}
I20260927 07:09:33.477265 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.473262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.4.174.66:36533)
0927 07:09:33.473463 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.473468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.473479 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:09:33.473517 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.473519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.473535 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.473620 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:33.473813 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.474954 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.476393 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.476882 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.476945 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.477019 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.477135 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.477170 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":78,"threads_started":1}
I20260927 07:09:33.477495 12799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.473109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39549 (local address 127.0.0.1:41492)
0927 07:09:33.473291 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.473333 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.473413 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.473876 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.473887 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.473889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:33.473940 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.473942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.475155 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.475156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.476145 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.476149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.477251 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.477253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.477330 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.477335 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.477384 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.477423 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":41,"threads_started":1}
I20260927 07:09:33.477645 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.473639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.4.174.65:42435)
0927 07:09:33.473823 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.473827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.474121 (+ 294us) server_negotiation.cc:407] Connection header received
0927 07:09:33.474165 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.474168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.474181 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.474241 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:33.474353 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.475109 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.476411 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.477190 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.477252 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.477325 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.477509 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.477550 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260927 07:09:33.477805 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.473403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.4.174.67:39549)
0927 07:09:33.473653 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.473657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.473665 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:33.473709 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.473711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.473724 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.473792 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:33.474017 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.475074 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.476213 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.477112 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.477573 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.477654 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.477678 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.477714 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":67,"threads_started":1}
I20260927 07:09:33.477582 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.473282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42435 (local address 127.0.0.1:33598)
0927 07:09:33.474024 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.474033 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.474073 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.474248 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.474249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.474250 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:33.474295 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.474297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.475194 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.475195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.476322 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.476323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.477357 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.477358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.477432 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.477436 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.477477 (+ 41us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.477507 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":10,"thread_start_us":61,"threads_started":1}
I20260927 07:09:33.478503 12710 tablet_service.cc:1467] Processing CreateTablet for tablet 508a9d925bef40ab8a587378d93f1462 (DEFAULT_TABLE table=test-table [id=25bc116b7d2f4f3c892467f885a13cde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.478508 12570 tablet_service.cc:1467] Processing CreateTablet for tablet 508a9d925bef40ab8a587378d93f1462 (DEFAULT_TABLE table=test-table [id=25bc116b7d2f4f3c892467f885a13cde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.478508 12423 tablet_service.cc:1467] Processing CreateTablet for tablet 508a9d925bef40ab8a587378d93f1462 (DEFAULT_TABLE table=test-table [id=25bc116b7d2f4f3c892467f885a13cde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.478878 12570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 508a9d925bef40ab8a587378d93f1462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.478883 12423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 508a9d925bef40ab8a587378d93f1462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.478879 12710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 508a9d925bef40ab8a587378d93f1462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.481827 12804 tablet_bootstrap.cc:492] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: Bootstrap starting.
I20260927 07:09:33.482543 12805 tablet_bootstrap.cc:492] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: Bootstrap starting.
I20260927 07:09:33.482594 12804 tablet_bootstrap.cc:654] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.482931 12804 log.cc:824] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:33.483134 12805 tablet_bootstrap.cc:654] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.483270 12806 tablet_bootstrap.cc:492] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Bootstrap starting.
I20260927 07:09:33.483457 12805 log.cc:824] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:33.483810 12806 tablet_bootstrap.cc:654] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.484122 12806 log.cc:824] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:33.484159 12805 tablet_bootstrap.cc:492] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: No bootstrap required, opened a new log
I20260927 07:09:33.484211 12805 ts_tablet_manager.cc:1397] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:33.484622 12804 tablet_bootstrap.cc:492] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: No bootstrap required, opened a new log
I20260927 07:09:33.484692 12804 ts_tablet_manager.cc:1397] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 07:09:33.484769 12806 tablet_bootstrap.cc:492] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: No bootstrap required, opened a new log
I20260927 07:09:33.484834 12806 ts_tablet_manager.cc:1397] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:33.486189 12806 raft_consensus.cc:348] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
I20260927 07:09:33.486315 12806 raft_consensus.cc:374] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.486344 12806 raft_consensus.cc:729] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edc62022738427ea3884a4b90f606d1, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.486472 12806 consensus_queue.cc:260] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
I20260927 07:09:33.486418 12805 raft_consensus.cc:348] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
I20260927 07:09:33.486634 12805 raft_consensus.cc:374] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.486604 12804 raft_consensus.cc:348] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
I20260927 07:09:33.486701 12805 raft_consensus.cc:729] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75992d841c246f18ca0753f9c571d9b, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.486766 12806 ts_tablet_manager.cc:1428] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:33.486788 12636 heartbeater.cc:503] Master 127.4.174.126:41397 was elected leader, sending a full tablet report...
I20260927 07:09:33.486829 12804 raft_consensus.cc:374] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.486814 12805 consensus_queue.cc:260] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
I20260927 07:09:33.486871 12804 raft_consensus.cc:729] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6b951e028141a38b91443f73cd8aaf, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.486989 12804 consensus_queue.cc:260] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
I20260927 07:09:33.487046 12805 ts_tablet_manager.cc:1428] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:09:33.487146 12496 heartbeater.cc:503] Master 127.4.174.126:41397 was elected leader, sending a full tablet report...
I20260927 07:09:33.487465 12804 ts_tablet_manager.cc:1428] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:09:33.487502 12776 heartbeater.cc:503] Master 127.4.174.126:41397 was elected leader, sending a full tablet report...
I20260927 07:09:33.516409 12810 raft_consensus.cc:482] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:33.516551 12810 raft_consensus.cc:504] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
I20260927 07:09:33.516880 12810 leader_election.cc:288] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549), b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435)
I20260927 07:09:33.520686 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.517335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:50275 (local address 127.4.174.65:42435)
0927 07:09:33.517400 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.517402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.517466 (+ 64us) server_negotiation.cc:407] Connection header received
0927 07:09:33.517541 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.517542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.517550 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.517604 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:33.517727 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.518416 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.519749 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.520309 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.520380 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.520390 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.520578 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.520615 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 07:09:33.520686 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.517230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:35001 (local address 127.4.174.67:39549)
0927 07:09:33.517281 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.517283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.517292 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:33.517321 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.517322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.517331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.517415 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:33.517567 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.518300 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.519751 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.520297 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.520386 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.520395 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.520579 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.520617 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 07:09:33.520726 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.517116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39549 (local address 127.4.174.66:35001)
0927 07:09:33.517175 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.517188 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.517236 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.517423 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.517424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.517427 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:33.517502 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.517505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.518396 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.518397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.519643 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.519645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.520396 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.520397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.520498 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.520502 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.520568 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.520611 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":8}
I20260927 07:09:33.520744 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.517234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42435 (local address 127.4.174.66:50275)
0927 07:09:33.517416 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.517425 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.517478 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.517619 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.517620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.517621 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:33.517674 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.517676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.518487 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.518487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.519638 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.519643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.520402 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.520403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.520498 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.520503 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.520568 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.520611 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":4,"thread_start_us":118,"threads_started":1}
I20260927 07:09:33.521088 12730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "508a9d925bef40ab8a587378d93f1462" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b951e028141a38b91443f73cd8aaf" is_pre_election: true
I20260927 07:09:33.521121 12450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "508a9d925bef40ab8a587378d93f1462" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75992d841c246f18ca0753f9c571d9b" is_pre_election: true
I20260927 07:09:33.521236 12730 raft_consensus.cc:2454] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 0.
I20260927 07:09:33.521274 12450 raft_consensus.cc:2454] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 0.
I20260927 07:09:33.521462 12524 leader_election.cc:302] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b951e028141a38b91443f73cd8aaf, 1edc62022738427ea3884a4b90f606d1; no voters:
I20260927 07:09:33.521601 12810 raft_consensus.cc:2781] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:33.521667 12810 raft_consensus.cc:482] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:33.521687 12810 raft_consensus.cc:3037] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.522666 12810 raft_consensus.cc:504] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
I20260927 07:09:33.522783 12810 leader_election.cc:288] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 election: Requested vote from peers 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549), b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435)
I20260927 07:09:33.522951 12730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "508a9d925bef40ab8a587378d93f1462" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b951e028141a38b91443f73cd8aaf"
I20260927 07:09:33.522961 12450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "508a9d925bef40ab8a587378d93f1462" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75992d841c246f18ca0753f9c571d9b"
I20260927 07:09:33.523028 12450 raft_consensus.cc:3037] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.523030 12730 raft_consensus.cc:3037] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.523686 12730 raft_consensus.cc:2454] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 1.
I20260927 07:09:33.523731 12450 raft_consensus.cc:2454] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 1.
I20260927 07:09:33.523849 12524 leader_election.cc:302] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b951e028141a38b91443f73cd8aaf, 1edc62022738427ea3884a4b90f606d1; no voters:
I20260927 07:09:33.523934 12810 raft_consensus.cc:2781] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:33.524094 12810 raft_consensus.cc:686] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 1 LEADER]: Becoming Leader. State: Replica: 1edc62022738427ea3884a4b90f606d1, State: Running, Role: LEADER
I20260927 07:09:33.524183 12810 consensus_queue.cc:237] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } }
W20260927 07:09:33.524255 12637 tablet.cc:2369] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:33.525058 12293 catalog_manager.cc:5495] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1edc62022738427ea3884a4b90f606d1 (127.4.174.66). New cstate: current_term: 1 leader_uuid: "1edc62022738427ea3884a4b90f606d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:33.535008 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.532654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.4.174.66:36533)
0927 07:09:33.532709 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.532711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.532720 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:33.532769 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.532770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.532785 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.532839 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:33.532945 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.533441 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.534204 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.534406 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.534486 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.534691 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:33.534807 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:33.534870 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.534939 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.534957 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:33.536726 12450 raft_consensus.cc:1261] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Refusing update from remote peer 1edc62022738427ea3884a4b90f606d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:33.536726 12730 raft_consensus.cc:1261] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Refusing update from remote peer 1edc62022738427ea3884a4b90f606d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:33.537077 12814 consensus_queue.cc:1035] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:33.537194 12810 consensus_queue.cc:1035] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:33.539454 12817 mvcc.cc:205] Tried to move back new op lower bound from 7333859219604017152 to 7333859219555774464. Current Snapshot: MvccSnapshot[applied={T|T < 7333859219604017152}]
I20260927 07:09:33.540069 12819 mvcc.cc:205] Tried to move back new op lower bound from 7333859219604017152 to 7333859219555774464. Current Snapshot: MvccSnapshot[applied={T|T < 7333859219604017152}]
I20260927 07:09:33.542160 12292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:33.542265 12292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:33.543033 12292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: Sending DeleteTablet for 3 replicas of tablet 508a9d925bef40ab8a587378d93f1462
I20260927 07:09:33.543340 12570 tablet_service.cc:1514] Processing DeleteTablet for tablet 508a9d925bef40ab8a587378d93f1462 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:33 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:53912
I20260927 07:09:33.543383 12710 tablet_service.cc:1514] Processing DeleteTablet for tablet 508a9d925bef40ab8a587378d93f1462 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:33 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:41492
I20260927 07:09:33.543566 12826 tablet_replica.cc:330] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: stopping tablet replica
I20260927 07:09:33.543561 12423 tablet_service.cc:1514] Processing DeleteTablet for tablet 508a9d925bef40ab8a587378d93f1462 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:33 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:33598
I20260927 07:09:33.543673 12826 raft_consensus.cc:2229] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:33.543736 12828 tablet_replica.cc:330] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: stopping tablet replica
I20260927 07:09:33.543813 12826 raft_consensus.cc:2258] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:33.543850 12828 raft_consensus.cc:2229] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:33.543992 12828 raft_consensus.cc:2258] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:33.544113 12292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:33.544260 12292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:33.544273 12826 ts_tablet_manager.cc:1905] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:33.544638 12827 tablet_replica.cc:330] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: stopping tablet replica
I20260927 07:09:33.544767 12827 raft_consensus.cc:2229] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:33.544902 12828 ts_tablet_manager.cc:1905] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:33.545161 12827 raft_consensus.cc:2258] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:33.545644 12827 ts_tablet_manager.cc:1905] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:33.545810 12570 tablet_service.cc:1467] Processing CreateTablet for tablet f5535b1275d941f0b1ebca7d373568e9 (DEFAULT_TABLE table=test-table [id=190fb39800804037bca83dd370b40580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.545922 12570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5535b1275d941f0b1ebca7d373568e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.546878 12828 ts_tablet_manager.cc:1918] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:33.546938 12828 log.cc:1196] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/508a9d925bef40ab8a587378d93f1462
I20260927 07:09:33.547160 12827 ts_tablet_manager.cc:1918] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:33.547171 12828 ts_tablet_manager.cc:1939] T 508a9d925bef40ab8a587378d93f1462 P b75992d841c246f18ca0753f9c571d9b: Deleting consensus metadata
I20260927 07:09:33.547217 12827 log.cc:1196] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/508a9d925bef40ab8a587378d93f1462
I20260927 07:09:33.547251 12806 tablet_bootstrap.cc:492] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: Bootstrap starting.
I20260927 07:09:33.547610 12280 catalog_manager.cc:4841] TS b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435): tablet 508a9d925bef40ab8a587378d93f1462 (table test-table [id=25bc116b7d2f4f3c892467f885a13cde]) successfully deleted
I20260927 07:09:33.547624 12423 tablet_service.cc:1467] Processing CreateTablet for tablet f5535b1275d941f0b1ebca7d373568e9 (DEFAULT_TABLE table=test-table [id=190fb39800804037bca83dd370b40580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.547645 12827 ts_tablet_manager.cc:1939] T 508a9d925bef40ab8a587378d93f1462 P 0c6b951e028141a38b91443f73cd8aaf: Deleting consensus metadata
I20260927 07:09:33.547669 12806 tablet_bootstrap.cc:654] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.547711 12423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5535b1275d941f0b1ebca7d373568e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.547797 12710 tablet_service.cc:1467] Processing CreateTablet for tablet f5535b1275d941f0b1ebca7d373568e9 (DEFAULT_TABLE table=test-table [id=190fb39800804037bca83dd370b40580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.547884 12710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5535b1275d941f0b1ebca7d373568e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.548003 12280 catalog_manager.cc:4841] TS 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549): tablet 508a9d925bef40ab8a587378d93f1462 (table test-table [id=25bc116b7d2f4f3c892467f885a13cde]) successfully deleted
I20260927 07:09:33.548626 12805 tablet_bootstrap.cc:492] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: Bootstrap starting.
I20260927 07:09:33.548957 12805 tablet_bootstrap.cc:654] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.549022 12804 tablet_bootstrap.cc:492] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: Bootstrap starting.
I20260927 07:09:33.549372 12804 tablet_bootstrap.cc:654] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.549528 12805 tablet_bootstrap.cc:492] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: No bootstrap required, opened a new log
I20260927 07:09:33.549566 12805 ts_tablet_manager.cc:1397] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:33.549705 12805 raft_consensus.cc:348] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.549763 12805 raft_consensus.cc:374] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.549779 12805 raft_consensus.cc:729] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75992d841c246f18ca0753f9c571d9b, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.549817 12805 consensus_queue.cc:260] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.549891 12805 ts_tablet_manager.cc:1428] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.551456 12826 ts_tablet_manager.cc:1918] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:33.551527 12826 log.cc:1196] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/508a9d925bef40ab8a587378d93f1462
I20260927 07:09:33.551748 12826 ts_tablet_manager.cc:1939] T 508a9d925bef40ab8a587378d93f1462 P 1edc62022738427ea3884a4b90f606d1: Deleting consensus metadata
I20260927 07:09:33.552258 12278 catalog_manager.cc:4841] TS 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533): tablet 508a9d925bef40ab8a587378d93f1462 (table test-table [id=25bc116b7d2f4f3c892467f885a13cde]) successfully deleted
I20260927 07:09:33.552699 12804 tablet_bootstrap.cc:492] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: No bootstrap required, opened a new log
I20260927 07:09:33.552738 12804 ts_tablet_manager.cc:1397] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 07:09:33.552868 12804 raft_consensus.cc:348] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.552924 12804 raft_consensus.cc:374] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.552942 12804 raft_consensus.cc:729] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6b951e028141a38b91443f73cd8aaf, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.552986 12804 consensus_queue.cc:260] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.553058 12804 ts_tablet_manager.cc:1428] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:33.558940 12806 tablet_bootstrap.cc:492] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: No bootstrap required, opened a new log
I20260927 07:09:33.559003 12806 ts_tablet_manager.cc:1397] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260927 07:09:33.559163 12806 raft_consensus.cc:348] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.559244 12806 raft_consensus.cc:374] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.559263 12806 raft_consensus.cc:729] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edc62022738427ea3884a4b90f606d1, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.559305 12806 consensus_queue.cc:260] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.559392 12806 ts_tablet_manager.cc:1428] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.591460 12814 raft_consensus.cc:482] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:33.591576 12814 raft_consensus.cc:504] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.591741 12814 leader_election.cc:288] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435), 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549)
I20260927 07:09:33.591931 12730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5535b1275d941f0b1ebca7d373568e9" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b951e028141a38b91443f73cd8aaf" is_pre_election: true
I20260927 07:09:33.592046 12730 raft_consensus.cc:2454] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 0.
I20260927 07:09:33.592049 12450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5535b1275d941f0b1ebca7d373568e9" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75992d841c246f18ca0753f9c571d9b" is_pre_election: true
I20260927 07:09:33.592147 12450 raft_consensus.cc:2454] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 0.
I20260927 07:09:33.592185 12524 leader_election.cc:302] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b951e028141a38b91443f73cd8aaf, 1edc62022738427ea3884a4b90f606d1; no voters:
I20260927 07:09:33.592275 12814 raft_consensus.cc:2781] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:33.592314 12814 raft_consensus.cc:482] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:33.592336 12814 raft_consensus.cc:3037] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.593264 12814 raft_consensus.cc:504] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.593411 12814 leader_election.cc:288] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 election: Requested vote from peers b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435), 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549)
I20260927 07:09:33.593564 12450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5535b1275d941f0b1ebca7d373568e9" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75992d841c246f18ca0753f9c571d9b"
I20260927 07:09:33.593580 12730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5535b1275d941f0b1ebca7d373568e9" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b951e028141a38b91443f73cd8aaf"
I20260927 07:09:33.593647 12450 raft_consensus.cc:3037] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.593652 12730 raft_consensus.cc:3037] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.594348 12730 raft_consensus.cc:2454] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 1.
I20260927 07:09:33.594439 12450 raft_consensus.cc:2454] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 1.
I20260927 07:09:33.594506 12524 leader_election.cc:302] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b951e028141a38b91443f73cd8aaf, 1edc62022738427ea3884a4b90f606d1; no voters:
I20260927 07:09:33.594627 12814 raft_consensus.cc:2781] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:33.594681 12814 raft_consensus.cc:686] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 1 LEADER]: Becoming Leader. State: Replica: 1edc62022738427ea3884a4b90f606d1, State: Running, Role: LEADER
I20260927 07:09:33.594730 12814 consensus_queue.cc:237] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.595278 12292 catalog_manager.cc:5495] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1edc62022738427ea3884a4b90f606d1 (127.4.174.66). New cstate: current_term: 1 leader_uuid: "1edc62022738427ea3884a4b90f606d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } health_report { overall_health: UNKNOWN } } }
W20260927 07:09:33.629169 12497 tablet.cc:2369] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492972, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.126@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:33.635003 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.627371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48860 (local address 127.4.174.126:41397)
0927 07:09:33.627431 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.627433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.627502 (+ 69us) server_negotiation.cc:407] Connection header received
0927 07:09:33.627573 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.627574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.627587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.627653 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.628080 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.628834 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.629647 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.629925 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.630012 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.632388 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.632390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.632390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.632398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.633925 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.634248 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.634249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.634249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.634297 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.634472 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.634473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.634473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.634614 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.634709 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.634803 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.634882 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:33.635066 12838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.627285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41397 (local address 127.0.0.1:48860)
0927 07:09:33.627425 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.627436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.627497 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.627656 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.627657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.627935 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:33.627996 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.627998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.628939 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.628940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.629563 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.629565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.630028 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.630029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.630081 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.630734 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:33.630738 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:33.632286 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:33.633994 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.634003 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.634005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.634188 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.634380 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:33.634380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:33.634380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:33.634430 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:33.634669 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:33.634677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:33.634723 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.634781 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.634943 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":59,"thread_start_us":68,"threads_started":1}
I20260927 07:09:33.637476 12292 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:48860:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 07:09:33.637661 12292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:33.638945 12570 tablet_service.cc:1467] Processing CreateTablet for tablet afa8fde99dcb4b70acf33b7ce70d37dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d355ac9b56d43d8bfda12b5f3813659]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:33.639003 12423 tablet_service.cc:1467] Processing CreateTablet for tablet afa8fde99dcb4b70acf33b7ce70d37dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d355ac9b56d43d8bfda12b5f3813659]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:33.639007 12710 tablet_service.cc:1467] Processing CreateTablet for tablet afa8fde99dcb4b70acf33b7ce70d37dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d355ac9b56d43d8bfda12b5f3813659]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:09:33.639109 12570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa8fde99dcb4b70acf33b7ce70d37dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.639143 12710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa8fde99dcb4b70acf33b7ce70d37dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.639143 12423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa8fde99dcb4b70acf33b7ce70d37dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.640578 12805 tablet_bootstrap.cc:492] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: Bootstrap starting.
I20260927 07:09:33.640573 12804 tablet_bootstrap.cc:492] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: Bootstrap starting.
I20260927 07:09:33.640674 12806 tablet_bootstrap.cc:492] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: Bootstrap starting.
I20260927 07:09:33.640971 12804 tablet_bootstrap.cc:654] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.640971 12805 tablet_bootstrap.cc:654] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.641108 12806 tablet_bootstrap.cc:654] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.641526 12805 tablet_bootstrap.cc:492] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: No bootstrap required, opened a new log
I20260927 07:09:33.641570 12805 ts_tablet_manager.cc:1397] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:33.641733 12804 tablet_bootstrap.cc:492] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: No bootstrap required, opened a new log
I20260927 07:09:33.641768 12804 ts_tablet_manager.cc:1397] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:33.641726 12805 raft_consensus.cc:348] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.641803 12805 raft_consensus.cc:374] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.641834 12805 raft_consensus.cc:729] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75992d841c246f18ca0753f9c571d9b, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.641875 12805 consensus_queue.cc:260] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.641891 12804 raft_consensus.cc:348] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.641952 12804 raft_consensus.cc:374] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.641978 12805 ts_tablet_manager.cc:1428] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:33.641978 12804 raft_consensus.cc:729] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6b951e028141a38b91443f73cd8aaf, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.642030 12804 consensus_queue.cc:260] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.642110 12804 ts_tablet_manager.cc:1428] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.642112 12811 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.642179 12811 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:33.642230 12812 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.642299 12812 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:33.646978 12806 tablet_bootstrap.cc:492] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: No bootstrap required, opened a new log
I20260927 07:09:33.647027 12806 ts_tablet_manager.cc:1397] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260927 07:09:33.647158 12806 raft_consensus.cc:348] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.647224 12806 raft_consensus.cc:374] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.647236 12806 raft_consensus.cc:729] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edc62022738427ea3884a4b90f606d1, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.647266 12806 consensus_queue.cc:260] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.647353 12806 ts_tablet_manager.cc:1428] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.647392 12814 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.647503 12814 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260927 07:09:33.681594 12777 tablet.cc:2369] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:33.758505 12812 raft_consensus.cc:482] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:33.758628 12812 raft_consensus.cc:504] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.759017 12812 leader_election.cc:288] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435), 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533)
I20260927 07:09:33.762804 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.759260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:56119 (local address 127.4.174.66:36533)
0927 07:09:33.759303 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.759304 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.759493 (+ 189us) server_negotiation.cc:407] Connection header received
0927 07:09:33.759528 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.759529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.759544 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.759615 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:33.759723 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.760399 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.761965 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.762502 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.762592 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.762601 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.762700 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.762740 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:09:33.762804 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.759207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:40645 (local address 127.4.174.65:42435)
0927 07:09:33.759257 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.759259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.759270 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:09:33.759326 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.759327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.759337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.759395 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:33.759539 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.760327 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.761764 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.762444 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.762520 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.762531 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.762690 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.762730 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 07:09:33.762822 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.759149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42435 (local address 127.4.174.67:40645)
0927 07:09:33.759207 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.759218 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.759262 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.759401 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.759402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.759404 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:33.759475 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.759477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.760419 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.760420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.761686 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.761689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.762517 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.762518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.762615 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.762619 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.762678 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.762723 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 07:09:33.762833 12842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.759184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36533 (local address 127.4.174.67:56119)
0927 07:09:33.759424 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.759433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.759483 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.759617 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.759617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.759618 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:33.759668 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.759669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.760474 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.760475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.761855 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.761857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.762582 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.762583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.762641 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.762644 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.762690 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.762721 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":14,"thread_start_us":154,"threads_started":1}
I20260927 07:09:33.763247 12590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa8fde99dcb4b70acf33b7ce70d37dd" candidate_uuid: "0c6b951e028141a38b91443f73cd8aaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1edc62022738427ea3884a4b90f606d1" is_pre_election: true
I20260927 07:09:33.763392 12590 raft_consensus.cc:2454] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6b951e028141a38b91443f73cd8aaf in term 0.
I20260927 07:09:33.763355 12450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa8fde99dcb4b70acf33b7ce70d37dd" candidate_uuid: "0c6b951e028141a38b91443f73cd8aaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75992d841c246f18ca0753f9c571d9b" is_pre_election: true
I20260927 07:09:33.763466 12450 raft_consensus.cc:2454] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6b951e028141a38b91443f73cd8aaf in term 0.
I20260927 07:09:33.763582 12662 leader_election.cc:302] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b951e028141a38b91443f73cd8aaf, 1edc62022738427ea3884a4b90f606d1; no voters:
I20260927 07:09:33.763731 12812 raft_consensus.cc:2781] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:33.763772 12812 raft_consensus.cc:482] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:33.763790 12812 raft_consensus.cc:3037] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.764456 12812 raft_consensus.cc:504] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.764567 12812 leader_election.cc:288] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [CANDIDATE]: Term 1 election: Requested vote from peers b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435), 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533)
I20260927 07:09:33.764724 12590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa8fde99dcb4b70acf33b7ce70d37dd" candidate_uuid: "0c6b951e028141a38b91443f73cd8aaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1edc62022738427ea3884a4b90f606d1"
I20260927 07:09:33.764791 12590 raft_consensus.cc:3037] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.764739 12450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa8fde99dcb4b70acf33b7ce70d37dd" candidate_uuid: "0c6b951e028141a38b91443f73cd8aaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75992d841c246f18ca0753f9c571d9b"
I20260927 07:09:33.764827 12450 raft_consensus.cc:3037] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:33.765372 12590 raft_consensus.cc:2454] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6b951e028141a38b91443f73cd8aaf in term 1.
I20260927 07:09:33.765484 12450 raft_consensus.cc:2454] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6b951e028141a38b91443f73cd8aaf in term 1.
I20260927 07:09:33.765519 12662 leader_election.cc:302] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b951e028141a38b91443f73cd8aaf, 1edc62022738427ea3884a4b90f606d1; no voters:
I20260927 07:09:33.765609 12812 raft_consensus.cc:2781] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:33.765753 12812 raft_consensus.cc:686] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [term 1 LEADER]: Becoming Leader. State: Replica: 0c6b951e028141a38b91443f73cd8aaf, State: Running, Role: LEADER
I20260927 07:09:33.765820 12812 consensus_queue.cc:237] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.766227 12843 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: TxnStatusTablet state changed. Reason: New leader 0c6b951e028141a38b91443f73cd8aaf. Latest consensus state: current_term: 1 leader_uuid: "0c6b951e028141a38b91443f73cd8aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.766299 12843 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:33.766465 12845 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 07:09:33.766587 12845 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 07:09:33.766592 12292 catalog_manager.cc:5495] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67). New cstate: current_term: 1 leader_uuid: "0c6b951e028141a38b91443f73cd8aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } health_report { overall_health: HEALTHY } } }
I20260927 07:09:33.791652 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.789312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.4.174.67:39549)
0927 07:09:33.789354 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.789356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.789384 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:09:33.789479 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.789480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.789499 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.789554 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:33.789672 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.790155 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.790930 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.791121 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.791170 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.791330 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:33.791443 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:33.791505 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.791549 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.791569 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:33.791661 12838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.789273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39549 (local address 127.0.0.1:42538)
0927 07:09:33.789333 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.789346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.789405 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.789564 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.789565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.789567 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:33.789616 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.789619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.790224 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.790225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.790852 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.790855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.791174 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.791175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.791238 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.791252 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:33.791497 (+ 245us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:33.791505 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.791547 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.791593 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 07:09:33.792935 12590 raft_consensus.cc:1261] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Refusing update from remote peer 0c6b951e028141a38b91443f73cd8aaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:33.792927 12450 raft_consensus.cc:1261] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Refusing update from remote peer 0c6b951e028141a38b91443f73cd8aaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:33.793205 12843 consensus_queue.cc:1035] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:33.793327 12812 consensus_queue.cc:1035] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:33.793856 12811 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: TxnStatusTablet state changed. Reason: New leader 0c6b951e028141a38b91443f73cd8aaf. Latest consensus state: current_term: 1 leader_uuid: "0c6b951e028141a38b91443f73cd8aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.793951 12811 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:33.793893 12814 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: TxnStatusTablet state changed. Reason: New leader 0c6b951e028141a38b91443f73cd8aaf. Latest consensus state: current_term: 1 leader_uuid: "0c6b951e028141a38b91443f73cd8aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.794003 12814 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:33.794811 12812 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c6b951e028141a38b91443f73cd8aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.794896 12812 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:33.794943 12812 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c6b951e028141a38b91443f73cd8aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.794981 12812 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P 0c6b951e028141a38b91443f73cd8aaf: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:09:33.795238 12814 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c6b951e028141a38b91443f73cd8aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.795310 12814 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P 1edc62022738427ea3884a4b90f606d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:33.796032 12818 mvcc.cc:205] Tried to move back new op lower bound from 7333859220653662208 to 7333859220545429504. Current Snapshot: MvccSnapshot[applied={T|T < 7333859220653662208}]
I20260927 07:09:33.796273 12811 tablet_replica.cc:439] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c6b951e028141a38b91443f73cd8aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } } }
I20260927 07:09:33.796361 12811 tablet_replica.cc:442] T afa8fde99dcb4b70acf33b7ce70d37dd P b75992d841c246f18ca0753f9c571d9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:09:33.800985 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.798686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:58267 (local address 127.4.174.67:39549)
0927 07:09:33.798725 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.798727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.798842 (+ 115us) server_negotiation.cc:407] Connection header received
0927 07:09:33.798891 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.798892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.798902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.798970 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:33.799066 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.799599 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.800332 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.800551 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.800604 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.800758 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:33.800793 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:33.800836 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.800888 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.800904 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:09:33.801002 12854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.798649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39549 (local address 127.4.174.66:58267)
0927 07:09:33.798782 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.798791 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.798845 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.798971 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.798972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.798974 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:33.799021 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.799023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.799664 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.799665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.800256 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.800258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.800612 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.800613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.800673 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.800689 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:33.800841 (+ 152us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:33.800842 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.800878 (+ 36us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.800932 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":53,"threads_started":1}
I20260927 07:09:33.802943 12450 raft_consensus.cc:1261] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Refusing update from remote peer 1edc62022738427ea3884a4b90f606d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:33.803010 12730 raft_consensus.cc:1261] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Refusing update from remote peer 1edc62022738427ea3884a4b90f606d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:33.803166 12810 consensus_queue.cc:1035] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:33.803273 12810 consensus_queue.cc:1035] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:33.811466 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.809152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:43475 (local address 127.4.174.66:36533)
0927 07:09:33.809205 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.809207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.809215 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:33.809255 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.809256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.809267 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.809311 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:33.809442 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.809909 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.810691 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.810952 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.811007 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.811225 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:33.811264 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:33.811306 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.811380 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.811399 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 07:09:33.811524 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.809082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36533 (local address 127.4.174.67:43475)
0927 07:09:33.809159 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:33.809170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:33.809207 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:33.809314 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:33.809315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:33.809316 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:09:33.809381 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.809382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.809987 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.809988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.810624 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:33.810626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.811024 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.811025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:33.811112 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.811135 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:09:33.811321 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:09:33.811322 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 07:09:33.811379 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 07:09:33.811429 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 07:09:33.817685 12292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:33.817785 12292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:33.818331 12292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: Sending DeleteTablet for 3 replicas of tablet f5535b1275d941f0b1ebca7d373568e9
I20260927 07:09:33.818619 12570 tablet_service.cc:1514] Processing DeleteTablet for tablet f5535b1275d941f0b1ebca7d373568e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:33 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:53912
I20260927 07:09:33.818722 12826 tablet_replica.cc:330] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: stopping tablet replica
I20260927 07:09:33.818758 12423 tablet_service.cc:1514] Processing DeleteTablet for tablet f5535b1275d941f0b1ebca7d373568e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:33 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:33598
I20260927 07:09:33.818804 12826 raft_consensus.cc:2229] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:33.818837 12828 tablet_replica.cc:330] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: stopping tablet replica
I20260927 07:09:33.818862 12826 raft_consensus.cc:2258] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:33.818873 12828 raft_consensus.cc:2229] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:33.818902 12828 raft_consensus.cc:2258] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:33.819178 12828 ts_tablet_manager.cc:1905] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:33.819214 12826 ts_tablet_manager.cc:1905] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:33.820636 12828 ts_tablet_manager.cc:1918] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:33.820698 12828 log.cc:1196] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/f5535b1275d941f0b1ebca7d373568e9
I20260927 07:09:33.820876 12826 ts_tablet_manager.cc:1918] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:33.820896 12828 ts_tablet_manager.cc:1939] T f5535b1275d941f0b1ebca7d373568e9 P b75992d841c246f18ca0753f9c571d9b: Deleting consensus metadata
I20260927 07:09:33.820942 12826 log.cc:1196] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/f5535b1275d941f0b1ebca7d373568e9
I20260927 07:09:33.821163 12826 ts_tablet_manager.cc:1939] T f5535b1275d941f0b1ebca7d373568e9 P 1edc62022738427ea3884a4b90f606d1: Deleting consensus metadata
I20260927 07:09:33.821194 12280 catalog_manager.cc:4841] TS b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435): tablet f5535b1275d941f0b1ebca7d373568e9 (table test-table [id=190fb39800804037bca83dd370b40580]) successfully deleted
I20260927 07:09:33.821444 12278 catalog_manager.cc:4841] TS 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533): tablet f5535b1275d941f0b1ebca7d373568e9 (table test-table [id=190fb39800804037bca83dd370b40580]) successfully deleted
I20260927 07:09:33.824143 12710 tablet_service.cc:1514] Processing DeleteTablet for tablet f5535b1275d941f0b1ebca7d373568e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:33 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:41492
I20260927 07:09:33.824278 12827 tablet_replica.cc:330] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: stopping tablet replica
I20260927 07:09:33.824333 12827 raft_consensus.cc:2229] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:33.824378 12827 raft_consensus.cc:2258] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:33.824647 12827 ts_tablet_manager.cc:1905] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:33.825891 12827 ts_tablet_manager.cc:1918] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:33.825943 12827 log.cc:1196] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/f5535b1275d941f0b1ebca7d373568e9
I20260927 07:09:33.826120 12827 ts_tablet_manager.cc:1939] T f5535b1275d941f0b1ebca7d373568e9 P 0c6b951e028141a38b91443f73cd8aaf: Deleting consensus metadata
I20260927 07:09:33.826460 12280 catalog_manager.cc:4841] TS 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549): tablet f5535b1275d941f0b1ebca7d373568e9 (table test-table [id=190fb39800804037bca83dd370b40580]) successfully deleted
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:09:33.846793 12345 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:33.841312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48690 (local address 127.4.174.126:41397)
0927 07:09:33.841364 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.841366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.841504 (+ 138us) server_negotiation.cc:407] Connection header received
0927 07:09:33.841572 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.841573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.841584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.841630 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.842192 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.842725 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.843457 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.843652 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.843706 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.846721 (+ 3015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48690: BlockingRecv error: recv got EOF from 127.0.0.1:48690 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 27 07:09:33 dist-test-slave-zw7p krb5kdc[12248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM
I20260927 07:09:33.855213 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:33.847661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48704 (local address 127.4.174.126:41397)
0927 07:09:33.847705 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:33.847707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:33.847719 (+ 12us) server_negotiation.cc:407] Connection header received
0927 07:09:33.847795 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:33.847796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:33.847808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:33.847882 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:33.848283 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.848991 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.849675 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:33.849959 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:33.850040 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:33.852385 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:33.852387 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:33.852387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:33.852396 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:33.854038 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.854368 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.854369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.854370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.854440 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:33.854638 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:33.854640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:33.854640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:33.854751 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:33.855010 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:33.855068 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 07:09:33.855138 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:33.856832 12292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:33.857012 12292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:33.858474 12423 tablet_service.cc:1467] Processing CreateTablet for tablet c9e45ce991f34cd58a07ae77b3dc1f68 (DEFAULT_TABLE table=test-table [id=34e27880811d47678777cbb290de4ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.858498 12570 tablet_service.cc:1467] Processing CreateTablet for tablet c9e45ce991f34cd58a07ae77b3dc1f68 (DEFAULT_TABLE table=test-table [id=34e27880811d47678777cbb290de4ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.858503 12710 tablet_service.cc:1467] Processing CreateTablet for tablet c9e45ce991f34cd58a07ae77b3dc1f68 (DEFAULT_TABLE table=test-table [id=34e27880811d47678777cbb290de4ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:33.858654 12710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e45ce991f34cd58a07ae77b3dc1f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.858654 12570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e45ce991f34cd58a07ae77b3dc1f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.858668 12423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e45ce991f34cd58a07ae77b3dc1f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:33.859948 12806 tablet_bootstrap.cc:492] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: Bootstrap starting.
I20260927 07:09:33.860141 12805 tablet_bootstrap.cc:492] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: Bootstrap starting.
I20260927 07:09:33.860267 12806 tablet_bootstrap.cc:654] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.860508 12804 tablet_bootstrap.cc:492] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: Bootstrap starting.
I20260927 07:09:33.860524 12805 tablet_bootstrap.cc:654] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.860916 12804 tablet_bootstrap.cc:654] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:33.861085 12805 tablet_bootstrap.cc:492] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: No bootstrap required, opened a new log
I20260927 07:09:33.861123 12805 ts_tablet_manager.cc:1397] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:33.861261 12805 raft_consensus.cc:348] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.861327 12805 raft_consensus.cc:374] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.861346 12805 raft_consensus.cc:729] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75992d841c246f18ca0753f9c571d9b, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.861402 12805 consensus_queue.cc:260] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.861493 12805 ts_tablet_manager.cc:1428] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.861661 12806 tablet_bootstrap.cc:492] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: No bootstrap required, opened a new log
I20260927 07:09:33.861706 12806 ts_tablet_manager.cc:1397] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:33.861805 12806 raft_consensus.cc:348] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.861863 12806 raft_consensus.cc:374] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.861881 12806 raft_consensus.cc:729] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edc62022738427ea3884a4b90f606d1, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.861939 12806 consensus_queue.cc:260] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.861996 12804 tablet_bootstrap.cc:492] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: No bootstrap required, opened a new log
I20260927 07:09:33.862026 12806 ts_tablet_manager.cc:1428] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:33.862031 12804 ts_tablet_manager.cc:1397] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:33.862309 12804 raft_consensus.cc:348] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.862381 12804 raft_consensus.cc:374] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:33.862419 12804 raft_consensus.cc:729] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6b951e028141a38b91443f73cd8aaf, State: Initialized, Role: FOLLOWER
I20260927 07:09:33.862455 12804 consensus_queue.cc:260] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:33.862540 12804 ts_tablet_manager.cc:1428] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:34.092643 12811 raft_consensus.cc:482] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:34.092742 12811 raft_consensus.cc:504] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.093063 12811 leader_election.cc:288] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533), 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549)
I20260927 07:09:34.096069 12875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:34.093231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39549 (local address 127.4.174.65:48625)
0927 07:09:34.093347 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:34.093357 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:34.093420 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:34.093555 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:34.093555 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:34.093558 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:34.093639 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:34.093641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.094332 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.094332 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:34.095192 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:34.095195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.095803 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.095804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:34.095870 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:34.095874 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:34.095929 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 07:09:34.095970 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260927 07:09:34.096081 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:34.093296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:48625 (local address 127.4.174.67:39549)
0927 07:09:34.093354 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:34.093356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:34.093421 (+ 65us) server_negotiation.cc:407] Connection header received
0927 07:09:34.093486 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:34.093488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:34.093496 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:34.093542 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:34.093696 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.094231 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.095276 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.095720 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.095771 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:34.095779 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:34.095976 (+ 197us) server_negotiation.cc:299] Negotiation successful
0927 07:09:34.096014 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:34.096338 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:34.093309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:60889 (local address 127.4.174.66:36533)
0927 07:09:34.093366 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:34.093368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:34.093458 (+ 90us) server_negotiation.cc:407] Connection header received
0927 07:09:34.093514 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:34.093515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:34.093527 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:34.093578 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:34.093712 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.094168 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.095530 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.095976 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.096039 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:34.096047 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:34.096235 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 07:09:34.096288 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 07:09:34.096318 12730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e45ce991f34cd58a07ae77b3dc1f68" candidate_uuid: "b75992d841c246f18ca0753f9c571d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b951e028141a38b91443f73cd8aaf" is_pre_election: true
I20260927 07:09:34.096415 12730 raft_consensus.cc:2454] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b75992d841c246f18ca0753f9c571d9b in term 0.
I20260927 07:09:34.096582 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:34.093231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36533 (local address 127.4.174.65:60889)
0927 07:09:34.093378 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:34.093388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:34.093464 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:34.093587 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:34.093587 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:34.093590 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:34.093653 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:34.093654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.094233 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.094234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:34.095441 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:34.095444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.096056 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.096058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:34.096132 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:34.096136 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:34.096451 (+ 315us) client_negotiation.cc:240] Negotiation successful
0927 07:09:34.096494 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":36,"thread_start_us":42,"threads_started":1}
I20260927 07:09:34.096593 12384 leader_election.cc:302] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b951e028141a38b91443f73cd8aaf, b75992d841c246f18ca0753f9c571d9b; no voters:
I20260927 07:09:34.096737 12811 raft_consensus.cc:2781] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:34.096781 12811 raft_consensus.cc:482] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:34.096798 12811 raft_consensus.cc:3037] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:34.096828 12590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e45ce991f34cd58a07ae77b3dc1f68" candidate_uuid: "b75992d841c246f18ca0753f9c571d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1edc62022738427ea3884a4b90f606d1" is_pre_election: true
I20260927 07:09:34.096928 12590 raft_consensus.cc:2454] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b75992d841c246f18ca0753f9c571d9b in term 0.
I20260927 07:09:34.097543 12811 raft_consensus.cc:504] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.097658 12811 leader_election.cc:288] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [CANDIDATE]: Term 1 election: Requested vote from peers 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533), 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549)
I20260927 07:09:34.097817 12730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e45ce991f34cd58a07ae77b3dc1f68" candidate_uuid: "b75992d841c246f18ca0753f9c571d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b951e028141a38b91443f73cd8aaf"
I20260927 07:09:34.097890 12730 raft_consensus.cc:3037] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:34.097812 12590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e45ce991f34cd58a07ae77b3dc1f68" candidate_uuid: "b75992d841c246f18ca0753f9c571d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1edc62022738427ea3884a4b90f606d1"
I20260927 07:09:34.097930 12590 raft_consensus.cc:3037] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:34.098624 12590 raft_consensus.cc:2454] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b75992d841c246f18ca0753f9c571d9b in term 1.
I20260927 07:09:34.098624 12730 raft_consensus.cc:2454] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b75992d841c246f18ca0753f9c571d9b in term 1.
I20260927 07:09:34.098825 12382 leader_election.cc:302] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1edc62022738427ea3884a4b90f606d1, b75992d841c246f18ca0753f9c571d9b; no voters:
I20260927 07:09:34.098918 12811 raft_consensus.cc:2781] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:34.099036 12811 raft_consensus.cc:686] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 1 LEADER]: Becoming Leader. State: Replica: b75992d841c246f18ca0753f9c571d9b, State: Running, Role: LEADER
I20260927 07:09:34.099123 12811 consensus_queue.cc:237] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.099910 12292 catalog_manager.cc:5495] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to b75992d841c246f18ca0753f9c571d9b (127.4.174.65). New cstate: current_term: 1 leader_uuid: "b75992d841c246f18ca0753f9c571d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:34.146240 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:34.143982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.4.174.65:42435)
0927 07:09:34.144034 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:34.144035 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:34.144043 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:34.144088 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:34.144089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:34.144114 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:34.144182 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:34.144290 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.144780 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.145517 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.145724 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.145778 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:34.145919 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:34.146029 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:34.146091 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:34.146140 (+ 49us) server_negotiation.cc:299] Negotiation successful
0927 07:09:34.146160 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 07:09:34.147208 12730 raft_consensus.cc:1261] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Refusing update from remote peer b75992d841c246f18ca0753f9c571d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:34.147212 12590 raft_consensus.cc:1261] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Refusing update from remote peer b75992d841c246f18ca0753f9c571d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:34.147467 12877 consensus_queue.cc:1035] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:34.147598 12811 consensus_queue.cc:1035] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:34.150426 12293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:34.150518 12293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:34.151156 12293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: Sending DeleteTablet for 3 replicas of tablet c9e45ce991f34cd58a07ae77b3dc1f68
I20260927 07:09:34.151396 12423 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e45ce991f34cd58a07ae77b3dc1f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:34 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:33598
I20260927 07:09:34.151414 12570 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e45ce991f34cd58a07ae77b3dc1f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:34 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:53912
I20260927 07:09:34.151484 12828 tablet_replica.cc:330] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: stopping tablet replica
I20260927 07:09:34.151525 12828 raft_consensus.cc:2229] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:34.151602 12828 raft_consensus.cc:2258] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:34.151670 12826 tablet_replica.cc:330] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: stopping tablet replica
I20260927 07:09:34.151623 12293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 07:09:34.151727 12826 raft_consensus.cc:2229] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260927 07:09:34.151760 12293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:34.151770 12826 raft_consensus.cc:2258] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:34.152088 12826 ts_tablet_manager.cc:1905] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:34.152164 12710 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e45ce991f34cd58a07ae77b3dc1f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:34 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:41492
I20260927 07:09:34.152246 12827 tablet_replica.cc:330] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: stopping tablet replica
I20260927 07:09:34.152294 12827 raft_consensus.cc:2229] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:34.152341 12827 raft_consensus.cc:2258] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:34.152633 12827 ts_tablet_manager.cc:1905] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:34.153276 12570 tablet_service.cc:1467] Processing CreateTablet for tablet 72ebf38f61f949bea09d52238897b345 (DEFAULT_TABLE table=test-table [id=98c16a733043430d8d63286ff31a4b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:34.153378 12570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72ebf38f61f949bea09d52238897b345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:34.153707 12826 ts_tablet_manager.cc:1918] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:34.153754 12826 log.cc:1196] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/c9e45ce991f34cd58a07ae77b3dc1f68
I20260927 07:09:34.153962 12826 ts_tablet_manager.cc:1939] T c9e45ce991f34cd58a07ae77b3dc1f68 P 1edc62022738427ea3884a4b90f606d1: Deleting consensus metadata
I20260927 07:09:34.154057 12827 ts_tablet_manager.cc:1918] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:34.154106 12827 log.cc:1196] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/c9e45ce991f34cd58a07ae77b3dc1f68
I20260927 07:09:34.154285 12827 ts_tablet_manager.cc:1939] T c9e45ce991f34cd58a07ae77b3dc1f68 P 0c6b951e028141a38b91443f73cd8aaf: Deleting consensus metadata
I20260927 07:09:34.154505 12806 tablet_bootstrap.cc:492] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: Bootstrap starting.
I20260927 07:09:34.154820 12806 tablet_bootstrap.cc:654] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:34.155144 12278 catalog_manager.cc:4841] TS 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533): tablet c9e45ce991f34cd58a07ae77b3dc1f68 (table test-table [id=34e27880811d47678777cbb290de4ce1]) successfully deleted
I20260927 07:09:34.155308 12280 catalog_manager.cc:4841] TS 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549): tablet c9e45ce991f34cd58a07ae77b3dc1f68 (table test-table [id=34e27880811d47678777cbb290de4ce1]) successfully deleted
I20260927 07:09:34.155540 12423 tablet_service.cc:1467] Processing CreateTablet for tablet 72ebf38f61f949bea09d52238897b345 (DEFAULT_TABLE table=test-table [id=98c16a733043430d8d63286ff31a4b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:34.155650 12423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72ebf38f61f949bea09d52238897b345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:34.155712 12806 tablet_bootstrap.cc:492] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: No bootstrap required, opened a new log
I20260927 07:09:34.155750 12806 ts_tablet_manager.cc:1397] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:34.155838 12806 raft_consensus.cc:348] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.155903 12806 raft_consensus.cc:374] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:34.155920 12806 raft_consensus.cc:729] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edc62022738427ea3884a4b90f606d1, State: Initialized, Role: FOLLOWER
I20260927 07:09:34.155958 12806 consensus_queue.cc:260] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.156033 12806 ts_tablet_manager.cc:1428] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:34.156595 12828 ts_tablet_manager.cc:1905] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:34.156917 12805 tablet_bootstrap.cc:492] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b: Bootstrap starting.
I20260927 07:09:34.157296 12805 tablet_bootstrap.cc:654] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:34.157667 12710 tablet_service.cc:1467] Processing CreateTablet for tablet 72ebf38f61f949bea09d52238897b345 (DEFAULT_TABLE table=test-table [id=98c16a733043430d8d63286ff31a4b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:34.157788 12710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72ebf38f61f949bea09d52238897b345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:34.157776 12828 ts_tablet_manager.cc:1918] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:34.157831 12828 log.cc:1196] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/c9e45ce991f34cd58a07ae77b3dc1f68
I20260927 07:09:34.158020 12828 ts_tablet_manager.cc:1939] T c9e45ce991f34cd58a07ae77b3dc1f68 P b75992d841c246f18ca0753f9c571d9b: Deleting consensus metadata
I20260927 07:09:34.158728 12805 tablet_bootstrap.cc:492] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b: No bootstrap required, opened a new log
I20260927 07:09:34.158771 12805 ts_tablet_manager.cc:1397] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:34.158934 12805 raft_consensus.cc:348] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.158996 12805 raft_consensus.cc:374] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:34.159014 12805 raft_consensus.cc:729] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75992d841c246f18ca0753f9c571d9b, State: Initialized, Role: FOLLOWER
I20260927 07:09:34.159025 12804 tablet_bootstrap.cc:492] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: Bootstrap starting.
I20260927 07:09:34.159050 12805 consensus_queue.cc:260] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.159142 12805 ts_tablet_manager.cc:1428] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:34.159313 12804 tablet_bootstrap.cc:654] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:34.159439 12280 catalog_manager.cc:4841] TS b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435): tablet c9e45ce991f34cd58a07ae77b3dc1f68 (table test-table [id=34e27880811d47678777cbb290de4ce1]) successfully deleted
I20260927 07:09:34.159864 12804 tablet_bootstrap.cc:492] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: No bootstrap required, opened a new log
I20260927 07:09:34.159914 12804 ts_tablet_manager.cc:1397] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:34.160020 12804 raft_consensus.cc:348] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.160089 12804 raft_consensus.cc:374] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:34.160116 12804 raft_consensus.cc:729] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6b951e028141a38b91443f73cd8aaf, State: Initialized, Role: FOLLOWER
I20260927 07:09:34.160154 12804 consensus_queue.cc:260] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.160239 12804 ts_tablet_manager.cc:1428] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:34.194501 12814 raft_consensus.cc:482] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:34.194621 12814 raft_consensus.cc:504] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.194782 12814 leader_election.cc:288] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435), 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549)
I20260927 07:09:34.194991 12730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72ebf38f61f949bea09d52238897b345" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b951e028141a38b91443f73cd8aaf" is_pre_election: true
I20260927 07:09:34.195014 12450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72ebf38f61f949bea09d52238897b345" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75992d841c246f18ca0753f9c571d9b" is_pre_election: true
I20260927 07:09:34.195083 12730 raft_consensus.cc:2454] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 0.
I20260927 07:09:34.195122 12450 raft_consensus.cc:2454] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 0.
I20260927 07:09:34.195248 12524 leader_election.cc:302] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b951e028141a38b91443f73cd8aaf, 1edc62022738427ea3884a4b90f606d1; no voters:
I20260927 07:09:34.195345 12814 raft_consensus.cc:2781] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:34.195384 12814 raft_consensus.cc:482] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:34.195405 12814 raft_consensus.cc:3037] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:34.196043 12814 raft_consensus.cc:504] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.196152 12814 leader_election.cc:288] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 election: Requested vote from peers b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435), 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549)
I20260927 07:09:34.196292 12450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72ebf38f61f949bea09d52238897b345" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75992d841c246f18ca0753f9c571d9b"
I20260927 07:09:34.196363 12450 raft_consensus.cc:3037] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:34.196357 12730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72ebf38f61f949bea09d52238897b345" candidate_uuid: "1edc62022738427ea3884a4b90f606d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b951e028141a38b91443f73cd8aaf"
I20260927 07:09:34.196431 12730 raft_consensus.cc:3037] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:34.196933 12450 raft_consensus.cc:2454] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 1.
I20260927 07:09:34.197078 12524 leader_election.cc:302] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1edc62022738427ea3884a4b90f606d1, b75992d841c246f18ca0753f9c571d9b; no voters:
I20260927 07:09:34.197166 12814 raft_consensus.cc:2781] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:34.197180 12730 raft_consensus.cc:2454] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1edc62022738427ea3884a4b90f606d1 in term 1.
I20260927 07:09:34.197208 12814 raft_consensus.cc:686] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 1 LEADER]: Becoming Leader. State: Replica: 1edc62022738427ea3884a4b90f606d1, State: Running, Role: LEADER
I20260927 07:09:34.197256 12814 consensus_queue.cc:237] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } }
I20260927 07:09:34.197808 12293 catalog_manager.cc:5495] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1edc62022738427ea3884a4b90f606d1 (127.4.174.66). New cstate: current_term: 1 leader_uuid: "1edc62022738427ea3884a4b90f606d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1edc62022738427ea3884a4b90f606d1" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 } health_report { overall_health: UNKNOWN } } }
I20260927 07:09:34.207617 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:34.205526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.4.174.66:36533)
0927 07:09:34.205572 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:34.205574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:34.205582 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:34.205627 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:34.205628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:34.205639 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:34.205677 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:34.205778 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.206257 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.207014 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:34.207213 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:34.207264 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:34.207387 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:34.207421 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:34.207463 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:34.207518 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 07:09:34.207541 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:34.209754 12450 raft_consensus.cc:1261] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Refusing update from remote peer 1edc62022738427ea3884a4b90f606d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:34.209946 12730 raft_consensus.cc:1261] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Refusing update from remote peer 1edc62022738427ea3884a4b90f606d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:34.209956 12814 consensus_queue.cc:1035] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75992d841c246f18ca0753f9c571d9b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:34.210155 12814 consensus_queue.cc:1035] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6b951e028141a38b91443f73cd8aaf" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:34.220160 12292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:34.220265 12292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:34.220850 12292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84d45cb3241c42f584ea251fe5398782: Sending DeleteTablet for 3 replicas of tablet 72ebf38f61f949bea09d52238897b345
I20260927 07:09:34.221087 12423 tablet_service.cc:1514] Processing DeleteTablet for tablet 72ebf38f61f949bea09d52238897b345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:34 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:33598
I20260927 07:09:34.221107 12570 tablet_service.cc:1514] Processing DeleteTablet for tablet 72ebf38f61f949bea09d52238897b345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:34 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:53912
I20260927 07:09:34.221184 12828 tablet_replica.cc:330] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b: stopping tablet replica
I20260927 07:09:34.221230 12828 raft_consensus.cc:2229] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:34.221262 12828 raft_consensus.cc:2258] T 72ebf38f61f949bea09d52238897b345 P b75992d841c246f18ca0753f9c571d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:34.221324 12826 tablet_replica.cc:330] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: stopping tablet replica
I20260927 07:09:34.221380 12826 raft_consensus.cc:2229] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:34.221428 12826 raft_consensus.cc:2258] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:34.221495 12710 tablet_service.cc:1514] Processing DeleteTablet for tablet 72ebf38f61f949bea09d52238897b345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:34 UTC) from {username='oryx', principal='oryx/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:41492
I20260927 07:09:34.221571 12827 tablet_replica.cc:330] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: stopping tablet replica
I20260927 07:09:34.221611 12827 raft_consensus.cc:2229] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:34.221644 12827 raft_consensus.cc:2258] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:34.221742 12826 ts_tablet_manager.cc:1905] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:34.221774 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 12365
I20260927 07:09:34.222396 12827 ts_tablet_manager.cc:1905] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:34.223476 12826 ts_tablet_manager.cc:1918] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:34.223526 12826 log.cc:1196] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/72ebf38f61f949bea09d52238897b345
I20260927 07:09:34.223748 12826 ts_tablet_manager.cc:1939] T 72ebf38f61f949bea09d52238897b345 P 1edc62022738427ea3884a4b90f606d1: Deleting consensus metadata
I20260927 07:09:34.223760 12827 ts_tablet_manager.cc:1918] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:09:34.223811 12827 log.cc:1196] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/72ebf38f61f949bea09d52238897b345
I20260927 07:09:34.224000 12827 ts_tablet_manager.cc:1939] T 72ebf38f61f949bea09d52238897b345 P 0c6b951e028141a38b91443f73cd8aaf: Deleting consensus metadata
I20260927 07:09:34.224088 12278 catalog_manager.cc:4841] TS 1edc62022738427ea3884a4b90f606d1 (127.4.174.66:36533): tablet 72ebf38f61f949bea09d52238897b345 (table test-table [id=98c16a733043430d8d63286ff31a4b02]) successfully deleted
I20260927 07:09:34.224323 12280 catalog_manager.cc:4841] TS 0c6b951e028141a38b91443f73cd8aaf (127.4.174.67:39549): tablet 72ebf38f61f949bea09d52238897b345 (table test-table [id=98c16a733043430d8d63286ff31a4b02]) successfully deleted
W20260927 07:09:34.228363 12280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:42435 (error 108)
I20260927 07:09:34.228547 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 12505
I20260927 07:09:34.228865 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:34.228706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42435 (local address 127.0.0.1:48650)
0927 07:09:34.228789 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:34.228825 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":36,"threads_started":1}
W20260927 07:09:34.228946 12280 catalog_manager.cc:4584] TS b75992d841c246f18ca0753f9c571d9b (127.4.174.65:42435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72ebf38f61f949bea09d52238897b345: Network error: Client connection negotiation failed: client connection to 127.4.174.65:42435: connect: Connection refused (error 111)
I20260927 07:09:34.234177 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 12645
I20260927 07:09:34.239316 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 12261
2026-09-27T07:09:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2494 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:34 dist-test-slave-zw7p krb5kdc[12894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:34 dist-test-slave-zw7p krb5kdc[12894](info): set up 2 sockets
Sep 27 07:09:34 dist-test-slave-zw7p krb5kdc[12894](info): commencing operation
krb5kdc: starting...
W20260927 07:09:35.182792 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:35 dist-test-slave-zw7p krb5kdc[12894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:35.291584 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:39853
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:39853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:35.368330 12907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:35.368534 12907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:35.368562 12907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:35.369675 12907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:35.369714 12907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:35.369733 12907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:35.369750 12907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:35.369776 12907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:35.371308 12907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:39853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:39853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:35.371543 12907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:35.371740 12907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:35.376216 12913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:35.376291 12915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:35.376224 12912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:35.376417 12907 server_base.cc:1034] running on GCE node
I20260927 07:09:35.376646 12907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:35.376907 12907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:35.378048 12907 hybrid_clock.cc:648] HybridClock initialized: now 1790492975378034 us; error 26 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-27T07:09:35Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1139 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:35 dist-test-slave-zw7p krb5kdc[12920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:35 dist-test-slave-zw7p krb5kdc[12920](info): set up 2 sockets
Sep 27 07:09:35 dist-test-slave-zw7p krb5kdc[12920](info): commencing operation
krb5kdc: starting...
W20260927 07:09:36.330834 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:36 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:36.440529 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:44821
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:44821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:36.517740 12933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:36.517937 12933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:36.517966 12933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:36.519160 12933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:36.519202 12933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:36.519222 12933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:36.519236 12933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:36.519253 12933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:36.520714 12933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:44821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:44821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:36.520947 12933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:36.521158 12933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:36.525657 12938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:36.525753 12939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:36.525766 12941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:36.525947 12933 server_base.cc:1034] running on GCE node
I20260927 07:09:36.526158 12933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:36.526407 12933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:36.527541 12933 hybrid_clock.cc:648] HybridClock initialized: now 1790492976527521 us; error 39 us; skew 500 ppm
Sep 27 07:09:36 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492976, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:36.529724 12933 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:36.530133 12933 webserver.cc:460] Webserver started at http://127.4.174.126:33941/ using document root <none> and password file <none>
I20260927 07:09:36.530334 12933 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:36.530380 12933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:36.530505 12933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:36.531603 12933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "003a2a5708b146d29e35b010a8a676d2"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "1e68003e406893db96cd9157a4a47e3e"
server_key_iv: "a101ebc4fdd0aa13f0ae77a18853013a"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.531908 12933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "003a2a5708b146d29e35b010a8a676d2"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "1e68003e406893db96cd9157a4a47e3e"
server_key_iv: "a101ebc4fdd0aa13f0ae77a18853013a"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.533107 12933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:36.533764 12948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:36.533919 12933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:36.533999 12933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "003a2a5708b146d29e35b010a8a676d2"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "1e68003e406893db96cd9157a4a47e3e"
server_key_iv: "a101ebc4fdd0aa13f0ae77a18853013a"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.534058 12933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:36.544234 12933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:36.545964 12933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:36.546090 12933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:36.549685 12933 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:44821
I20260927 07:09:36.549716 13000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:44821 every 8 connection(s)
I20260927 07:09:36.550031 12933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:36.550858 13001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:36.553041 13001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Bootstrap starting.
I20260927 07:09:36.553582 13001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:36.553833 13001 log.cc:824] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:36.554420 13001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: No bootstrap required, opened a new log
I20260927 07:09:36.554672 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 12933
I20260927 07:09:36.554785 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:36.554982 4793 external_mini_cluster.cc:1428] Setting key 34422a146a42b9f1bce7bb7d8e8e5414
I20260927 07:09:36.556595 13001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } }
I20260927 07:09:36.556774 13001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:36.556818 13001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003a2a5708b146d29e35b010a8a676d2, State: Initialized, Role: FOLLOWER
I20260927 07:09:36.556951 13001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } }
I20260927 07:09:36.557037 13001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:36.557080 13001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:36.557133 13001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:36.557651 13001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } }
I20260927 07:09:36.557744 13001 leader_election.cc:302] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 003a2a5708b146d29e35b010a8a676d2; no voters:
I20260927 07:09:36.557896 13001 leader_election.cc:288] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:36.557934 13006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:36.558074 13006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 1 LEADER]: Becoming Leader. State: Replica: 003a2a5708b146d29e35b010a8a676d2, State: Running, Role: LEADER
I20260927 07:09:36.558152 13001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:36.558213 13006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } }
I20260927 07:09:36.558599 13006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 003a2a5708b146d29e35b010a8a676d2. Latest consensus state: current_term: 1 leader_uuid: "003a2a5708b146d29e35b010a8a676d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } } }
I20260927 07:09:36.558710 13006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:36.558921 13015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:36.559401 13015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:36.559692 13008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "003a2a5708b146d29e35b010a8a676d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } } }
I20260927 07:09:36.559834 13008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: This master's current role is: LEADER
Sep 27 07:09:36 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:36.562191 13023 mvcc.cc:205] Tried to move back new op lower bound from 7333859231997050880 to 7333859231983009792. Current Snapshot: MvccSnapshot[applied={T|T < 7333859231997050880}]
I20260927 07:09:36.562661 13015 catalog_manager.cc:1347] Generated new cluster ID: 075e7a1ecc9a49a9a64ee2252e62dc9d
I20260927 07:09:36.562709 13015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:36.566285 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:36.555453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.4.174.126:44821)
0927 07:09:36.555624 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:36.555633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:36.555659 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:09:36.556155 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:36.556162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:36.556207 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:36.556316 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:36.557062 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.558482 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.559498 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.560120 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.560227 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:36.562681 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:36.562685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:36.562685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:36.562696 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:36.564523 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:36.564870 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:36.564873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:36.564873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:36.564961 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:36.565138 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:36.565138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:36.565139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:36.565397 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:36.565519 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:36.565946 (+ 427us) server_negotiation.cc:299] Negotiation successful
0927 07:09:36.566079 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260927 07:09:36.568957 13015 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:36.569422 13015 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:36.574366 13015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Generated new TSK 0
I20260927 07:09:36.574543 13015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:36.601320 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44821
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:36.679975 13027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:36.680194 13027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:36.680226 13027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:36.681396 13027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:36.681435 13027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:36.681470 13027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:36.682921 13027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:36.683163 13027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:36.683373 13027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:36.688079 13033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:36.688107 13032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:36.688128 13027 server_base.cc:1034] running on GCE node
W20260927 07:09:36.688107 13035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:36.688462 13027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:36.688688 13027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:36.689842 13027 hybrid_clock.cc:648] HybridClock initialized: now 1790492976689825 us; error 30 us; skew 500 ppm
Sep 27 07:09:36 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492976, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:36.697784 13027 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:36.698249 13027 webserver.cc:460] Webserver started at http://127.4.174.65:42639/ using document root <none> and password file <none>
I20260927 07:09:36.698462 13027 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:36.698505 13027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:36.698657 13027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:36.699733 13027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "532921be84444329bcba2db9084dabd4"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "a0d7882389d179a6fe61ed3c0a55355e"
server_key_iv: "15d44be522d86fc558ec0f8ac6b3235a"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.700062 13027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "532921be84444329bcba2db9084dabd4"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "a0d7882389d179a6fe61ed3c0a55355e"
server_key_iv: "15d44be522d86fc558ec0f8ac6b3235a"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.701247 13027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:36.701887 13042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:36.702025 13027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:36.702104 13027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "532921be84444329bcba2db9084dabd4"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "a0d7882389d179a6fe61ed3c0a55355e"
server_key_iv: "15d44be522d86fc558ec0f8ac6b3235a"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.702162 13027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:36.726769 13027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:36.728508 13027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:36.728641 13027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:36.728853 13027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:36.729156 13027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:36.729188 13027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:36.729216 13027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:36.729239 13027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:36.734803 13027 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:39593
I20260927 07:09:36.734854 13155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:39593 every 8 connection(s)
I20260927 07:09:36.735153 13027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:36.736104 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 13027
I20260927 07:09:36.736217 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:36.736395 4793 external_mini_cluster.cc:1428] Setting key 8afda209a3fb538cd44bc716207f1f74
Sep 27 07:09:36 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492976, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
I20260927 07:09:36.744828 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:36.735798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:60829 (local address 127.4.174.126:44821)
0927 07:09:36.735839 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:36.735841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:36.736051 (+ 210us) server_negotiation.cc:407] Connection header received
0927 07:09:36.736618 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:36.736620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:36.736633 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:36.736689 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:36.737706 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.738366 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.739615 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.739899 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.739981 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:36.742151 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:36.742153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:36.742153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:36.742162 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:36.743605 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:36.743987 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:36.743989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:36.743989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:36.744049 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:36.744254 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:36.744255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:36.744255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:36.744391 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:36.744455 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:36.744685 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 07:09:36.744759 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:09:36.745016 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:36.735779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:60829)
0927 07:09:36.735927 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:36.735972 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:36.736530 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:36.736779 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:36.736783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:36.737358 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:36.737613 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:36.737619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.738455 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.738456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:36.739530 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:36.739534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.740000 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.740001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:36.740118 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:36.740603 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:36.740611 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:36.742063 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:36.743685 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:36.743691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:36.743693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:36.743909 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:36.744107 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:36.744108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:36.744108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:36.744201 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:36.744446 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:36.744452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:36.744572 (+ 120us) client_negotiation.cc:769] Sending connection context
0927 07:09:36.744657 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:09:36.744780 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260927 07:09:36.745460 13156 heartbeater.cc:347] Connected to a master server at 127.4.174.126:44821
I20260927 07:09:36.745545 13156 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:36.745713 13156 heartbeater.cc:511] Master 127.4.174.126:44821 requested a full tablet report, sending...
I20260927 07:09:36.746244 12965 ts_manager.cc:194] Registered new tserver with Master: 532921be84444329bcba2db9084dabd4 (127.4.174.65:39593)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
I20260927 07:09:36.747067 12965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:60829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:36.759595 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44821
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:36.837445 13161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:36.837666 13161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:36.837697 13161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:36.838876 13161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:36.838914 13161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:36.838948 13161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:36.840375 13161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:36.840643 13161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:36.840891 13161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:36.846172 13167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:36.846184 13169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:36.846172 13166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:36.846524 13161 server_base.cc:1034] running on GCE node
I20260927 07:09:36.846704 13161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:36.846896 13161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:36.848026 13161 hybrid_clock.cc:648] HybridClock initialized: now 1790492976848017 us; error 35 us; skew 500 ppm
Sep 27 07:09:36 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492976, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:36.850345 13161 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:36.850827 13161 webserver.cc:460] Webserver started at http://127.4.174.66:44903/ using document root <none> and password file <none>
I20260927 07:09:36.851058 13161 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:36.851107 13161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:36.851243 13161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:36.852321 13161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "5965080ee6ad40d998296096fc995192"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "d6e4cc669782c4ad49a22e154b3d8071"
server_key_iv: "952ebe68bfa75961b7c600c626e75775"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.852666 13161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "5965080ee6ad40d998296096fc995192"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "d6e4cc669782c4ad49a22e154b3d8071"
server_key_iv: "952ebe68bfa75961b7c600c626e75775"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.854041 13161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:36.854739 13176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:36.854885 13161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:36.854960 13161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "5965080ee6ad40d998296096fc995192"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "d6e4cc669782c4ad49a22e154b3d8071"
server_key_iv: "952ebe68bfa75961b7c600c626e75775"
server_key_version: "encryptionkey@0"
I20260927 07:09:36.855027 13161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:36.874933 13161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:36.877008 13161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:36.877147 13161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:36.877393 13161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:36.877724 13161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:36.877789 13161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:36.877827 13161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:36.877856 13161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:36.883880 13161 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:43963
I20260927 07:09:36.883915 13289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:43963 every 8 connection(s)
I20260927 07:09:36.884222 13161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:09:36 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492976, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:36.892938 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:36.884914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:58525 (local address 127.4.174.126:44821)
0927 07:09:36.884956 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:36.884958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:36.885159 (+ 201us) server_negotiation.cc:407] Connection header received
0927 07:09:36.885655 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:36.885656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:36.885669 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:36.885714 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:36.886644 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.887149 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.888112 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.888298 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.888350 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:36.890341 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:36.890343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:36.890343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:36.890350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:36.891621 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:36.892034 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:36.892035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:36.892035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:36.892083 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:36.892339 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:36.892340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:36.892340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:36.892438 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:36.892487 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:36.892806 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 07:09:36.892869 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:09:36.893144 13292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:36.884889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:58525)
0927 07:09:36.885050 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:36.885085 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:36.885566 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:36.885787 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:36.885791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:36.886337 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:36.886540 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:36.886545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.887226 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.887227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:36.888034 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:36.888037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:36.888362 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:36.888363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:36.888495 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:36.888931 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:36.888939 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:36.890242 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:36.891706 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:36.891712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:36.891714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:36.891965 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:36.892155 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:36.892157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:36.892157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:36.892287 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:36.892541 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:36.892548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:36.892687 (+ 139us) client_negotiation.cc:769] Sending connection context
0927 07:09:36.892793 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 07:09:36.892924 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260927 07:09:36.893626 13290 heartbeater.cc:347] Connected to a master server at 127.4.174.126:44821
I20260927 07:09:36.893715 13290 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:36.893878 13290 heartbeater.cc:511] Master 127.4.174.126:44821 requested a full tablet report, sending...
I20260927 07:09:36.894248 12965 ts_manager.cc:194] Registered new tserver with Master: 5965080ee6ad40d998296096fc995192 (127.4.174.66:43963)
I20260927 07:09:36.894263 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 13161
I20260927 07:09:36.894356 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:36.894541 4793 external_mini_cluster.cc:1428] Setting key fccee64cbda8ee876388043f6117aa5b
I20260927 07:09:36.894724 12965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:58525
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:36.915616 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44821
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:36.996042 13295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:36.996254 13295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:36.996284 13295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:36.997427 13295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:36.997474 13295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:36.997510 13295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:36.998973 13295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:36.999209 13295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:36.999415 13295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:37.004163 13300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.004145 13303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.004132 13301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:37.004304 13295 server_base.cc:1034] running on GCE node
I20260927 07:09:37.004562 13295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:37.004789 13295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:37.005921 13295 hybrid_clock.cc:648] HybridClock initialized: now 1790492977005907 us; error 30 us; skew 500 ppm
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492977, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:37.008229 13295 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:37.008634 13295 webserver.cc:460] Webserver started at http://127.4.174.67:37925/ using document root <none> and password file <none>
I20260927 07:09:37.008844 13295 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:37.008888 13295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:37.009007 13295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:37.010077 13295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "56dac37e9ebb4856b0a9db20ed61ead4"
format_stamp: "Formatted at 2026-09-27 07:09:37 on dist-test-slave-zw7p"
server_key: "aa81e02208b5886ce13a72f8411f0ceb"
server_key_iv: "0d55610371d0706d15b86a5e191e5f79"
server_key_version: "encryptionkey@0"
I20260927 07:09:37.010406 13295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "56dac37e9ebb4856b0a9db20ed61ead4"
format_stamp: "Formatted at 2026-09-27 07:09:37 on dist-test-slave-zw7p"
server_key: "aa81e02208b5886ce13a72f8411f0ceb"
server_key_iv: "0d55610371d0706d15b86a5e191e5f79"
server_key_version: "encryptionkey@0"
I20260927 07:09:37.011611 13295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:37.012259 13310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.012398 13295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:37.012470 13295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "56dac37e9ebb4856b0a9db20ed61ead4"
format_stamp: "Formatted at 2026-09-27 07:09:37 on dist-test-slave-zw7p"
server_key: "aa81e02208b5886ce13a72f8411f0ceb"
server_key_iv: "0d55610371d0706d15b86a5e191e5f79"
server_key_version: "encryptionkey@0"
I20260927 07:09:37.012530 13295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:37.025617 13295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:37.027770 13295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:37.027915 13295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:37.028123 13295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:37.028411 13295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:37.028443 13295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.028466 13295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:37.028489 13295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.033921 13295 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:38285
I20260927 07:09:37.033977 13423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:38285 every 8 connection(s)
I20260927 07:09:37.034302 13295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492977, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:37.040242 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 13295
I20260927 07:09:37.040354 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:37.040534 4793 external_mini_cluster.cc:1428] Setting key 80abca08229fa246cb1058d26b3526c1
I20260927 07:09:37.043334 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.035030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:33393 (local address 127.4.174.126:44821)
0927 07:09:37.035105 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.035107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.035260 (+ 153us) server_negotiation.cc:407] Connection header received
0927 07:09:37.035728 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.035729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.035739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.035796 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.036776 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.037323 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.038285 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.038469 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.038514 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.040620 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:37.040622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:37.040623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:37.040631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:37.042069 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:37.042429 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:37.042431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:37.042431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:37.042489 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:37.042735 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:37.042736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:37.042737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:37.042856 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:37.042911 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:37.043166 (+ 255us) server_negotiation.cc:299] Negotiation successful
0927 07:09:37.043243 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:09:37.043524 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.034994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:33393)
0927 07:09:37.035140 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.035182 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.035644 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.035862 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.035865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.036483 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.036684 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.036689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.037392 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.037393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.038221 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.038223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.038523 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.038524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.038652 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.039133 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.039142 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.040538 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:37.042140 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:37.042147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:37.042149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:37.042367 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:37.042539 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:37.042540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:37.042540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:37.042680 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:37.042908 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:37.042914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:37.043054 (+ 140us) client_negotiation.cc:769] Sending connection context
0927 07:09:37.043141 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 07:09:37.043302 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260927 07:09:37.043946 13424 heartbeater.cc:347] Connected to a master server at 127.4.174.126:44821
I20260927 07:09:37.044035 13424 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:37.044179 13424 heartbeater.cc:511] Master 127.4.174.126:44821 requested a full tablet report, sending...
I20260927 07:09:37.044528 12964 ts_manager.cc:194] Registered new tserver with Master: 56dac37e9ebb4856b0a9db20ed61ead4 (127.4.174.67:38285)
I20260927 07:09:37.044871 12964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:33393
I20260927 07:09:37.052338 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab.
Entry for principal oryx/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab.
WARNING: no policy specified for oryx/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab.
Entry for principal oryx/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab.
WARNING: no policy specified for oryx/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab.
Entry for principal oryx/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab.
I20260927 07:09:37.119444 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 13027
I20260927 07:09:37.123301 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 13161
I20260927 07:09:37.127466 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 13295
I20260927 07:09:37.131079 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 12933
I20260927 07:09:37.135497 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:44821
--webserver_interface=127.4.174.126
--webserver_port=33941
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:44821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:37.214897 13433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:37.215092 13433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:37.215111 13433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:37.216281 13433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:37.216325 13433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:37.216338 13433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:37.216351 13433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:37.216363 13433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:37.217841 13433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:44821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:44821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=33941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:37.218067 13433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:37.218281 13433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:37.222872 13439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.222904 13438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.222879 13441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:37.223088 13433 server_base.cc:1034] running on GCE node
I20260927 07:09:37.223337 13433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:37.223672 13433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:37.224830 13433 hybrid_clock.cc:648] HybridClock initialized: now 1790492977224811 us; error 31 us; skew 500 ppm
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492977, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:37.227294 13433 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:37.227735 13433 webserver.cc:460] Webserver started at http://127.4.174.126:33941/ using document root <none> and password file <none>
I20260927 07:09:37.227967 13433 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:37.228034 13433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:37.229437 13433 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:37.230062 13448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.230201 13433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:37.230284 13433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "003a2a5708b146d29e35b010a8a676d2"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "1e68003e406893db96cd9157a4a47e3e"
server_key_iv: "a101ebc4fdd0aa13f0ae77a18853013a"
server_key_version: "encryptionkey@0"
I20260927 07:09:37.230661 13433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:37.254683 13433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:37.256858 13433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:37.257010 13433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:37.260730 13433 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:44821
I20260927 07:09:37.260769 13500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:44821 every 8 connection(s)
I20260927 07:09:37.261147 13433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:37.262598 13501 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:09:37.263196 13501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Bootstrap starting.
I20260927 07:09:37.269845 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 13433
I20260927 07:09:37.270133 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:39593
--local_ip_for_outbound_sockets=127.4.174.65
--tserver_master_addrs=127.4.174.126:44821
--webserver_port=42639
--webserver_interface=127.4.174.65
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.174.65
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:09:37.270673 13501 log.cc:824] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:37.272665 13501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:09:37.272907 13501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Bootstrap complete.
I20260927 07:09:37.275043 13501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } }
I20260927 07:09:37.275265 13501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003a2a5708b146d29e35b010a8a676d2, State: Initialized, Role: FOLLOWER
I20260927 07:09:37.275411 13501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } }
I20260927 07:09:37.275475 13501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:37.275516 13501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:37.275561 13501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:09:37.276538 13501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } }
I20260927 07:09:37.276626 13501 leader_election.cc:302] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 003a2a5708b146d29e35b010a8a676d2; no voters:
I20260927 07:09:37.276785 13501 leader_election.cc:288] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 07:09:37.276834 13505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 07:09:37.276950 13505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [term 2 LEADER]: Becoming Leader. State: Replica: 003a2a5708b146d29e35b010a8a676d2, State: Running, Role: LEADER
I20260927 07:09:37.277036 13505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } }
I20260927 07:09:37.277061 13501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:37.277593 13506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 003a2a5708b146d29e35b010a8a676d2. Latest consensus state: current_term: 2 leader_uuid: "003a2a5708b146d29e35b010a8a676d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } } }
I20260927 07:09:37.277671 13506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:37.277971 13507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "003a2a5708b146d29e35b010a8a676d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003a2a5708b146d29e35b010a8a676d2" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44821 } } }
I20260927 07:09:37.278028 13507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2 [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:37.278141 13514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:37.278673 13514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:37.278838 13514 catalog_manager.cc:1259] Loaded cluster ID: 075e7a1ecc9a49a9a64ee2252e62dc9d
I20260927 07:09:37.279029 13514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:37.280177 13514 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:37.280345 13514 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 003a2a5708b146d29e35b010a8a676d2: Loaded TSK: 0
I20260927 07:09:37.280548 13514 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 07:09:37.350652 13504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:37.350881 13504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:37.350905 13504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:37.352079 13504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:37.352125 13504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:37.352152 13504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:37.353554 13504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.65.keytab
--principal=oryx/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:39593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=42639
--tserver_master_addrs=127.4.174.126:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:37.353796 13504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:37.354027 13504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:37.359761 13528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.359843 13530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.359890 13527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:37.360069 13504 server_base.cc:1034] running on GCE node
I20260927 07:09:37.360251 13504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:37.360482 13504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:37.361642 13504 hybrid_clock.cc:648] HybridClock initialized: now 1790492977361623 us; error 31 us; skew 500 ppm
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492977, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:37.364151 13504 init.cc:377] Logged in from keytab as oryx/127.4.174.65@KRBTEST.COM (short username oryx)
I20260927 07:09:37.364584 13504 webserver.cc:460] Webserver started at http://127.4.174.65:42639/ using document root <none> and password file <none>
I20260927 07:09:37.364795 13504 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:37.364854 13504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:37.366160 13504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:37.366726 13537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.366899 13504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:37.367003 13504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "532921be84444329bcba2db9084dabd4"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "a0d7882389d179a6fe61ed3c0a55355e"
server_key_iv: "15d44be522d86fc558ec0f8ac6b3235a"
server_key_version: "encryptionkey@0"
I20260927 07:09:37.367426 13504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:37.379655 13504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:37.382144 13504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:37.382302 13504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:37.382580 13504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:37.382937 13504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:37.382997 13504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.383036 13504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:37.383059 13504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.389869 13504 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:39593
I20260927 07:09:37.389904 13650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:39593 every 8 connection(s)
I20260927 07:09:37.390314 13504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:37.395160 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 13504
I20260927 07:09:37.395478 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:43963
--local_ip_for_outbound_sockets=127.4.174.66
--tserver_master_addrs=127.4.174.126:44821
--webserver_port=44903
--webserver_interface=127.4.174.66
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.174.66
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:37.403424 13653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.391010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:41261)
0927 07:09:37.391159 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.391201 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.391663 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.395073 (+ 3410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.395078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.395826 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.396030 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.396036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.397500 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.397501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.398797 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.398801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.399678 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.399679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.399830 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.400463 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.400473 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.403220 (+ 2747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
W20260927 07:09:37.403861 13651 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:44821 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.174.126:44821: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
W20260927 07:09:37.404276 13654 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.391087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:41261 (local address 127.4.174.126:44821)
0927 07:09:37.394539 (+ 3452us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.394550 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.394633 (+ 83us) server_negotiation.cc:407] Connection header received
0927 07:09:37.394798 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.394804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.394853 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.394996 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.396164 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.397378 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.398904 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.399567 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.399695 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.403396 (+ 3701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:41261: BlockingRecv error: recv got EOF from 127.4.174.65:41261 (error 108)
Metrics: {"server-negotiator.queue_time_us":3367,"thread_start_us":98,"threads_started":1}
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:37.409404 13653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.404090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:55809)
0927 07:09:37.404139 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.404148 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.404186 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.404426 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.404427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.404562 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.404613 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.404614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.405580 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.405581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.406426 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.406429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.406885 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.406886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.406954 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.407377 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.407382 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.409332 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 07:09:37.409554 13656 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.404136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:55809 (local address 127.4.174.126:44821)
0927 07:09:37.404279 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.404281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.404288 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:37.404310 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.404310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.404321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.404384 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.404687 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.405482 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.406530 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.406809 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.406868 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.409428 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:55809: BlockingRecv error: recv got EOF from 127.4.174.65:55809 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:37.413764 13653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.409659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:43543)
0927 07:09:37.409699 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.409709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.409739 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.409885 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.409886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.409989 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.410017 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.410018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.410899 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.410899 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.411382 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.411383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.411790 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.411790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.411822 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.412217 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.412220 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.413719 (+ 1499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 07:09:37.413863 13656 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.409735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:43543 (local address 127.4.174.126:44821)
0927 07:09:37.409783 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.409785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.409793 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:37.409813 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.409814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.409824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.409878 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.410068 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.410840 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.411439 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.411736 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.411795 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.413810 (+ 2015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:43543: BlockingRecv error: recv got EOF from 127.4.174.65:43543 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260927 07:09:37.413887 13651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:09:37.479902 13655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:37.480110 13655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:37.480141 13655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:37.481310 13655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:37.481355 13655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:37.481389 13655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:37.482862 13655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.66.keytab
--principal=oryx/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:43963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=44903
--tserver_master_addrs=127.4.174.126:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:37.483114 13655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:37.483322 13655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:37.488140 13662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:37.488225 13655 server_base.cc:1034] running on GCE node
W20260927 07:09:37.488163 13661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.488158 13664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:37.488545 13655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:37.488791 13655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:37.489962 13655 hybrid_clock.cc:648] HybridClock initialized: now 1790492977489947 us; error 28 us; skew 500 ppm
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492977, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:37.492404 13655 init.cc:377] Logged in from keytab as oryx/127.4.174.66@KRBTEST.COM (short username oryx)
I20260927 07:09:37.492878 13655 webserver.cc:460] Webserver started at http://127.4.174.66:44903/ using document root <none> and password file <none>
I20260927 07:09:37.493089 13655 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:37.493152 13655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:37.494452 13655 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:37.495035 13671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.495182 13655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.495261 13655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "5965080ee6ad40d998296096fc995192"
format_stamp: "Formatted at 2026-09-27 07:09:36 on dist-test-slave-zw7p"
server_key: "d6e4cc669782c4ad49a22e154b3d8071"
server_key_iv: "952ebe68bfa75961b7c600c626e75775"
server_key_version: "encryptionkey@0"
I20260927 07:09:37.495622 13655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:37.531913 13655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:37.533732 13655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:37.533871 13655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:37.534108 13655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:37.534423 13655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:37.534464 13655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.534495 13655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:37.534519 13655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.539932 13655 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:43963
I20260927 07:09:37.539996 13784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:43963 every 8 connection(s)
I20260927 07:09:37.540323 13655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:37.540740 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 13655
I20260927 07:09:37.540997 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:38285
--local_ip_for_outbound_sockets=127.4.174.67
--tserver_master_addrs=127.4.174.126:44821
--webserver_port=37925
--webserver_interface=127.4.174.67
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.174.67
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:09:37.548079 13656 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.541202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:43583 (local address 127.4.174.126:44821)
0927 07:09:37.541260 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.541262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.541488 (+ 226us) server_negotiation.cc:407] Connection header received
0927 07:09:37.542129 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.542130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.542145 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.542211 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.543237 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.543736 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.544776 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.544969 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.545025 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.548018 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:43583: BlockingRecv error: recv got EOF from 127.4.174.66:43583 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:09:37.548144 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.541097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:43583)
0927 07:09:37.541337 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.541389 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.542010 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.542320 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.542324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.542932 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.543139 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.543144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.543817 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.543818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.544692 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.544695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.545038 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.545039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.545157 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.545704 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.545712 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.547915 (+ 2203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
W20260927 07:09:37.548352 13785 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:44821 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.174.126:44821: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:37.552912 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.548547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:34025)
0927 07:09:37.548669 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.548681 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.548736 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.548869 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.548869 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.549025 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.549087 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.549089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.549685 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.549686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.550517 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.550520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.550851 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.550852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.550917 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.551289 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.551293 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.552865 (+ 1572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103}
W20260927 07:09:37.553032 13656 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.548616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:34025 (local address 127.4.174.126:44821)
0927 07:09:37.548666 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.548668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.548720 (+ 52us) server_negotiation.cc:407] Connection header received
0927 07:09:37.548795 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.548795 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.548806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.548848 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.549150 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.549617 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.550619 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.550799 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.550841 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.552968 (+ 2127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:34025: BlockingRecv error: recv got EOF from 127.4.174.66:34025 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:09:37.557387 13656 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.553276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:36409 (local address 127.4.174.126:44821)
0927 07:09:37.553320 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.553322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.553330 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:37.553351 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.553352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.553360 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.553408 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.553607 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.554349 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.554995 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.555290 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.555350 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.557322 (+ 1972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:36409: BlockingRecv error: recv got EOF from 127.4.174.66:36409 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:09:37.557535 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.553221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:36409)
0927 07:09:37.553261 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.553270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.553302 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.553421 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.553421 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.553529 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.553556 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.553557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.554414 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.554414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.554935 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.554937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.555344 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.555345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.555389 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.555791 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.555795 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.557474 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260927 07:09:37.558122 13785 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:09:37.622354 13788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:37.622572 13788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:37.622597 13788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:37.623762 13788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:37.623802 13788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:37.623831 13788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:37.625214 13788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/krb5kdc/oryx_127.4.174.67.keytab
--principal=oryx/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:38285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=37925
--tserver_master_addrs=127.4.174.126:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:37.625414 13788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:37.625602 13788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:37.630187 13794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.630213 13796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:37.630321 13793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:37.630541 13788 server_base.cc:1034] running on GCE node
I20260927 07:09:37.630753 13788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:37.630971 13788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:37.632105 13788 hybrid_clock.cc:648] HybridClock initialized: now 1790492977632094 us; error 28 us; skew 500 ppm
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492977, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:37.634403 13788 init.cc:377] Logged in from keytab as oryx/127.4.174.67@KRBTEST.COM (short username oryx)
I20260927 07:09:37.634837 13788 webserver.cc:460] Webserver started at http://127.4.174.67:37925/ using document root <none> and password file <none>
I20260927 07:09:37.635047 13788 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:37.635111 13788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:37.636438 13788 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:37.636946 13803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.637084 13788 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.637168 13788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "56dac37e9ebb4856b0a9db20ed61ead4"
format_stamp: "Formatted at 2026-09-27 07:09:37 on dist-test-slave-zw7p"
server_key: "aa81e02208b5886ce13a72f8411f0ceb"
server_key_iv: "0d55610371d0706d15b86a5e191e5f79"
server_key_version: "encryptionkey@0"
I20260927 07:09:37.637476 13788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:37.656601 13788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:37.658643 13788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:37.658777 13788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:37.659006 13788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:37.659303 13788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:37.659337 13788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.659368 13788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:37.659392 13788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:37.664855 13788 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:38285
I20260927 07:09:37.664906 13916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:38285 every 8 connection(s)
I20260927 07:09:37.665234 13788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:37.665969 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 13788
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:09:37.673357 13656 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.665924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:33231 (local address 127.4.174.126:44821)
0927 07:09:37.665971 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.665972 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.666234 (+ 262us) server_negotiation.cc:407] Connection header received
0927 07:09:37.666930 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.666931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.666940 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.666983 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.668104 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.668837 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.670071 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.670357 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.670429 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.673302 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:33231: BlockingRecv error: recv got EOF from 127.4.174.67:33231 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:37.673563 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.665940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:33231)
0927 07:09:37.666102 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.666149 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.666842 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.667065 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.667069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.667760 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.668019 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.668025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.668914 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.668915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.669998 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.670001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.670442 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.670443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.670576 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.671019 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.671028 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.673216 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260927 07:09:37.673574 13654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.666431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.4.174.126:44821)
0927 07:09:37.666491 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.666493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.666541 (+ 48us) server_negotiation.cc:407] Connection header received
0927 07:09:37.666660 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.666661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.666677 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.666741 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.667272 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.668021 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.669056 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.669336 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.669399 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.670728 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:37.670734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:37.670734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:37.670748 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:37.672274 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:37.672580 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:37.672583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:37.672584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:37.672689 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:37.672855 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:37.672856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:37.672857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:37.673079 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:37.673210 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:37.673324 (+ 114us) server_negotiation.cc:299] Negotiation successful
0927 07:09:37.673428 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260927 07:09:37.673956 13917 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:44821 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.174.126:44821: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:37.677908 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.674219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:52225)
0927 07:09:37.674258 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.674267 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.674298 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.674468 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.674469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.674616 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.674666 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.674668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.675200 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.675201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.675693 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.675694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.675995 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.675996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.676037 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.676382 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.676386 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.677860 (+ 1474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260927 07:09:37.678040 13654 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.674257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:52225 (local address 127.4.174.126:44821)
0927 07:09:37.674293 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.674295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.674302 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:37.674393 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.674393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.674400 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.674453 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.674715 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.675151 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.675753 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.675946 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.675994 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.677947 (+ 1953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:52225: BlockingRecv error: recv got EOF from 127.4.174.67:52225 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:37 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:37.681947 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:37.678367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:35033)
0927 07:09:37.678418 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:37.678428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:37.678470 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:37.678611 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:37.678612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:37.678726 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:37.678754 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.678755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.679339 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.679340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.679844 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:37.679845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.680154 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.680155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:37.680203 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.680512 (+ 309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:37.680516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:37.681885 (+ 1369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 07:09:37.682014 13654 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:37.678430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:35033 (local address 127.4.174.126:44821)
0927 07:09:37.678465 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:37.678467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:37.678474 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:37.678523 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:37.678524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:37.678532 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:37.678606 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:37.678812 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.679258 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.679908 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:37.680100 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:37.680152 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:37.681966 (+ 1814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:35033: BlockingRecv error: recv got EOF from 127.4.174.67:35033 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 07:09:37.682102 13917 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 07:09:38 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:38 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:38.418721 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:38.414233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:54301)
0927 07:09:38.414350 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:38.414362 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:38.414409 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:38.414575 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:38.414576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:38.414734 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:38.414782 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:38.414784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.415395 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.415395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:38.415984 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:38.415987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.416301 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.416303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:38.416359 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:38.416868 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:38.416873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:38.418653 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
W20260927 07:09:38.418803 13922 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:38.414326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:54301 (local address 127.4.174.126:44821)
0927 07:09:38.414441 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:38.414443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:38.414451 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:38.414476 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:38.414477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:38.414486 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:38.414544 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:38.414838 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.415341 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.416043 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.416221 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.416264 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:38.418735 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:54301: BlockingRecv error: recv got EOF from 127.4.174.65:54301 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
Sep 27 07:09:38 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:38 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:38.562716 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:38.558470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:52457)
0927 07:09:38.558600 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:38.558611 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:38.558655 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:38.558763 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:38.558763 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:38.558907 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:38.558969 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:38.558971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.559574 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.559575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:38.560120 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:38.560121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.560400 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.560401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:38.560453 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:38.560859 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:38.560864 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:38.562646 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":53,"threads_started":1}
W20260927 07:09:38.562786 13922 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:38.558597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:52457 (local address 127.4.174.126:44821)
0927 07:09:38.558638 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:38.558639 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:38.558649 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:38.558696 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:38.558697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:38.558708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:38.558756 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:38.559018 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.559521 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.560170 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.560342 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.560383 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:38.562732 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:52457: BlockingRecv error: recv got EOF from 127.4.174.66:52457 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 27 07:09:38 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:38 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:38.686884 13924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:38.682485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:50503)
0927 07:09:38.682614 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:38.682627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:38.682698 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:38.682823 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:38.682824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:38.682979 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:38.683029 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:38.683031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.683670 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.683671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:38.684228 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:38.684229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.684522 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.684522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:38.684559 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:38.684994 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:38.684999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:38.686816 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
W20260927 07:09:38.687012 13922 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:38.682599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:50503 (local address 127.4.174.126:44821)
0927 07:09:38.682644 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:38.682646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:38.682680 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:09:38.682749 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:38.682750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:38.682762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:38.682809 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:38.683085 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.683581 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.684289 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:38.684472 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:38.684516 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:38.686907 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:50503: BlockingRecv error: recv got EOF from 127.4.174.67:50503 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 27 07:09:39 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:39 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:39.423758 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:39.419202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:52559)
0927 07:09:39.419318 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:39.419331 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:39.419417 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:39.419587 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:39.419588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:39.419746 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:39.419795 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:39.419797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.420496 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.420497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:39.421023 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:39.421024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.421400 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.421401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:39.421453 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:39.421895 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:39.421900 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:39.423685 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
W20260927 07:09:39.423916 13926 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:39.419295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:52559 (local address 127.4.174.126:44821)
0927 07:09:39.419470 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:39.419471 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:39.419483 (+ 12us) server_negotiation.cc:407] Connection header received
0927 07:09:39.419507 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:39.419507 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:39.419518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:39.419576 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:39.419852 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.420398 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.421102 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.421307 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.421358 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:39.423824 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:52559: BlockingRecv error: recv got EOF from 127.4.174.65:52559 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
Sep 27 07:09:39 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:39 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:39.567355 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:39.563102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:44947)
0927 07:09:39.563231 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:39.563242 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:39.563304 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:39.563442 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:39.563443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:39.563602 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:39.563650 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:39.563651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.564284 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.564285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:39.564806 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:39.564808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.565159 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.565160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:39.565214 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:39.565622 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:39.565626 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:39.567277 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
W20260927 07:09:39.567426 13926 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:39.563151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:44947 (local address 127.4.174.126:44821)
0927 07:09:39.563213 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:39.563214 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:39.563292 (+ 78us) server_negotiation.cc:407] Connection header received
0927 07:09:39.563359 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:39.563360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:39.563372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:39.563414 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:39.563714 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.564214 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.564879 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.565079 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.565134 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:39.567373 (+ 2239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:44947: BlockingRecv error: recv got EOF from 127.4.174.66:44947 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 27 07:09:39 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:39 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:39.691556 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:39.687302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:44349)
0927 07:09:39.687436 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:39.687448 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:39.687507 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:39.687637 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:39.687638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:39.687812 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:39.687859 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:39.687862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.688498 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.688500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:39.689017 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:39.689018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.689327 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.689328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:39.689379 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:39.689772 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:39.689777 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:39.691480 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
W20260927 07:09:39.691762 13926 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:39.687417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:44349 (local address 127.4.174.126:44821)
0927 07:09:39.687475 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:39.687477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:39.687489 (+ 12us) server_negotiation.cc:407] Connection header received
0927 07:09:39.687574 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:39.687574 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:39.687585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:39.687626 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:39.687935 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.688425 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.689086 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:39.689268 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:39.689322 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:39.691569 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:44349: BlockingRecv error: recv got EOF from 127.4.174.67:44349 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 27 07:09:40 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:40 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:40.428728 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:40.424237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:35921)
0927 07:09:40.424394 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:40.424406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:40.424459 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:40.424592 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:40.424593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:40.424738 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:40.424788 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:40.424790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.425411 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.425412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:40.425936 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:40.425938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.426260 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.426262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:40.426315 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:40.426781 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:40.426786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:40.428650 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
W20260927 07:09:40.428836 13930 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:40.424300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:35921 (local address 127.4.174.126:44821)
0927 07:09:40.424408 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:40.424410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:40.424448 (+ 38us) server_negotiation.cc:407] Connection header received
0927 07:09:40.424518 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:40.424518 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:40.424529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:40.424578 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:40.424839 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.425353 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.425995 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.426179 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.426224 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:40.428773 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:35921: BlockingRecv error: recv got EOF from 127.4.174.65:35921 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
Sep 27 07:09:40 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:40 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:40.572490 13931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:40.567971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:46913)
0927 07:09:40.568119 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:40.568131 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:40.568199 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:40.568327 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:40.568328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:40.568513 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:40.568562 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:40.568564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.569206 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.569207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:40.569731 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:40.569732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.570076 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.570076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:40.570131 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:40.570671 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:40.570676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:40.572411 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
W20260927 07:09:40.572582 13930 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:40.568031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:46913 (local address 127.4.174.126:44821)
0927 07:09:40.568093 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:40.568095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:40.568183 (+ 88us) server_negotiation.cc:407] Connection header received
0927 07:09:40.568252 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:40.568253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:40.568265 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:40.568350 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:40.568639 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.569128 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.569813 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.570013 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.570071 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:40.572527 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:46913: BlockingRecv error: recv got EOF from 127.4.174.66:46913 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 27 07:09:40 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:40 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:40.696199 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:40.692004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:51577)
0927 07:09:40.692123 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:40.692134 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:40.692209 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:40.692343 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:40.692343 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:40.692501 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:40.692551 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:40.692553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.693145 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.693146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:40.693650 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:40.693651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.693968 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.693970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:40.694021 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:40.694443 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:40.694448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:40.696136 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
W20260927 07:09:40.696305 13930 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:40.692077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:51577 (local address 127.4.174.126:44821)
0927 07:09:40.692161 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:40.692163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:40.692193 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:09:40.692271 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:40.692272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:40.692284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:40.692327 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:40.692601 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.693089 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.693701 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:40.693890 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:40.693935 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:40.696225 (+ 2290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:51577: BlockingRecv error: recv got EOF from 127.4.174.67:51577 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 27 07:09:41 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:41 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:41.433512 13933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:41.429190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:54135)
0927 07:09:41.429334 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:41.429345 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:41.429420 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:41.429540 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:41.429541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:41.429693 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:41.429741 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:41.429743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.430352 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.430353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:41.430946 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:41.430949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.431291 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.431292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:41.431333 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:41.431750 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:41.431754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:41.433422 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
W20260927 07:09:41.433559 13934 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:41.429267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:54135 (local address 127.4.174.126:44821)
0927 07:09:41.429369 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:41.429371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:41.429405 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:09:41.429473 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:41.429473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:41.429483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:41.429530 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:41.429799 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.430290 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.431022 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.431235 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.431287 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:41.433500 (+ 2213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:54135: BlockingRecv error: recv got EOF from 127.4.174.65:54135 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
Sep 27 07:09:41 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:41 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:41.577781 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:41.573087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:42977)
0927 07:09:41.573201 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:41.573211 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:41.573266 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:41.573398 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:41.573398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:41.573551 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:41.573600 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:41.573602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.574264 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.574265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:41.575059 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:41.575062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.575474 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.575476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:41.575537 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:41.576027 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:41.576031 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:41.577707 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
W20260927 07:09:41.577868 13934 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:41.573138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:42977 (local address 127.4.174.126:44821)
0927 07:09:41.573251 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:41.573253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:41.573265 (+ 12us) server_negotiation.cc:407] Connection header received
0927 07:09:41.573314 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:41.573315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:41.573326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:41.573384 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:41.573650 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.574173 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.575154 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.575402 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.575466 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:41.577815 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:42977: BlockingRecv error: recv got EOF from 127.4.174.66:42977 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 07:09:41 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:41 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:41.708890 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:41.696787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:39413)
0927 07:09:41.696933 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:41.696945 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:41.697003 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:41.697107 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:41.697108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:41.697265 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:41.697313 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:41.697315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.697984 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.697985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:41.698587 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:41.698589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.699008 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.699010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:41.699064 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:41.707138 (+ 8074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:41.707143 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:41.708831 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":59,"threads_started":1}
W20260927 07:09:41.709012 13934 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:41.696830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:39413 (local address 127.4.174.126:44821)
0927 07:09:41.696890 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:41.696892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:41.696997 (+ 105us) server_negotiation.cc:407] Connection header received
0927 07:09:41.697045 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:41.697046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:41.697058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:41.697102 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:41.697358 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.697899 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.698698 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:41.698920 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:41.698981 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:41.708938 (+ 9957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:39413: BlockingRecv error: recv got EOF from 127.4.174.67:39413 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 27 07:09:42 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:42 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:42.438186 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:42.433995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:44299)
0927 07:09:42.434130 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:42.434141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:42.434199 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:42.434375 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:42.434377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:42.434588 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:42.434643 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:42.434645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.435273 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.435273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:42.435793 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:42.435795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.436085 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.436085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:42.436119 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:42.436538 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:42.436543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:42.438128 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
W20260927 07:09:42.438295 13938 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:42.434041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:44299 (local address 127.4.174.126:44821)
0927 07:09:42.434167 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:42.434169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:42.434183 (+ 14us) server_negotiation.cc:407] Connection header received
0927 07:09:42.434279 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:42.434280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:42.434293 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:42.434356 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:42.434695 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.435209 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.435850 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.436036 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.436083 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:42.438219 (+ 2136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:44299: BlockingRecv error: recv got EOF from 127.4.174.65:44299 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
Sep 27 07:09:42 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:42 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:42.582636 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:42.578283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:33947)
0927 07:09:42.578425 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:42.578437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:42.578501 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:42.578639 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:42.578639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:42.578811 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:42.578857 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:42.578858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.579513 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.579514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:42.580055 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:42.580057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.580389 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.580390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:42.580445 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:42.580926 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:42.580930 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:42.582532 (+ 1602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
W20260927 07:09:42.582729 13938 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:42.578374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:33947 (local address 127.4.174.126:44821)
0927 07:09:42.578429 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:42.578431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:42.578482 (+ 51us) server_negotiation.cc:407] Connection header received
0927 07:09:42.578550 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:42.578551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:42.578588 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:42.578634 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:42.578928 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.579441 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.580135 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.580330 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.580384 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:42.582665 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:33947: BlockingRecv error: recv got EOF from 127.4.174.66:33947 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 27 07:09:42 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:42 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:42.714077 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:42.709359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:51353)
0927 07:09:42.709478 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:42.709488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:42.709550 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:42.709691 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:42.709692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:42.709845 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:42.709891 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:42.709892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.710881 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.710882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:42.711418 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:42.711420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.711884 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.711884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:42.711934 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:42.712339 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:42.712344 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:42.713996 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
W20260927 07:09:42.714151 13938 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:42.709433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:51353 (local address 127.4.174.126:44821)
0927 07:09:42.709485 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:42.709487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:42.709551 (+ 64us) server_negotiation.cc:407] Connection header received
0927 07:09:42.709603 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:42.709604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:42.709618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:42.709686 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:42.709968 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.710798 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.711502 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:42.711809 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:42.711874 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:42.714084 (+ 2210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:51353: BlockingRecv error: recv got EOF from 127.4.174.67:51353 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 27 07:09:43 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:43 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:43.445996 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:43.438638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:40379)
0927 07:09:43.438754 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:43.438764 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:43.438814 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:43.438935 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:43.438935 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:43.439083 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:43.439134 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:43.439135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.439795 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.439796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:43.440375 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:43.440378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.440745 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.440746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:43.440788 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:43.444241 (+ 3453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:43.444246 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:43.445933 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
W20260927 07:09:43.446172 13942 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:43.438706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:40379 (local address 127.4.174.126:44821)
0927 07:09:43.438812 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:43.438814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:43.438822 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:43.438858 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:43.438859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:43.438869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:43.438926 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:43.439188 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.439719 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.440454 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.440678 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.440732 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:43.446072 (+ 5340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:40379: BlockingRecv error: recv got EOF from 127.4.174.65:40379 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":48,"threads_started":1}
Sep 27 07:09:43 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:43 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:43.587397 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:43.583102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:60331)
0927 07:09:43.583238 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:43.583250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:43.583300 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:43.583437 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:43.583437 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:43.583593 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:43.583641 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:43.583642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.584283 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.584283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:43.584821 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:43.584823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.585161 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.585161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:43.585200 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:43.585620 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:43.585624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:43.587334 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
W20260927 07:09:43.587517 13942 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:43.583218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:60331 (local address 127.4.174.126:44821)
0927 07:09:43.583279 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:43.583281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:43.583351 (+ 70us) server_negotiation.cc:407] Connection header received
0927 07:09:43.583374 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:43.583374 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:43.583385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:43.583423 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:43.583724 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.584227 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.584904 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.585103 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.585156 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:43.587455 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:60331: BlockingRecv error: recv got EOF from 127.4.174.66:60331 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 27 07:09:43 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:43 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:43.718966 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:43.714605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:49293)
0927 07:09:43.714724 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:43.714735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:43.714802 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:43.714964 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:43.714965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:43.715119 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:43.715167 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:43.715169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.715883 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.715884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:43.716533 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:43.716535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.716850 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.716850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:43.716890 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:43.717279 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:43.717283 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:43.718909 (+ 1626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
W20260927 07:09:43.719064 13942 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:43.714677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:49293 (local address 127.4.174.126:44821)
0927 07:09:43.714736 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:43.714738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:43.714798 (+ 60us) server_negotiation.cc:407] Connection header received
0927 07:09:43.714845 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:43.714846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:43.714858 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:43.714918 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:43.715233 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.715790 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.716597 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:43.716791 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:43.716844 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:43.719001 (+ 2157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:49293: BlockingRecv error: recv got EOF from 127.4.174.67:49293 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 27 07:09:44 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:44 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:44.451210 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:44.446470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:38075)
0927 07:09:44.446618 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:44.446628 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:44.446678 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:44.446887 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:44.446888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:44.447050 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:44.447099 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:44.447101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.447815 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.447816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:44.448367 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:44.448369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.448735 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.448736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:44.448780 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:44.449232 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:44.449237 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:44.451144 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
W20260927 07:09:44.451298 13946 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:44.446626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:38075 (local address 127.4.174.126:44821)
0927 07:09:44.446747 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:44.446749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:44.446758 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:44.446782 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:44.446783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:44.446794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:44.446866 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:44.447177 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.447723 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.448462 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.448664 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.448729 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:44.451229 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:38075: BlockingRecv error: recv got EOF from 127.4.174.65:38075 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
Sep 27 07:09:44 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:44 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:44.592397 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:44.587925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:40959)
0927 07:09:44.588044 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:44.588056 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:44.588115 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:44.588250 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:44.588251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:44.588419 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:44.588468 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:44.588470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.589106 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.589107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:44.589639 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:44.589641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.589986 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.589987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:44.590044 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:44.590480 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:44.590484 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:44.592309 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
W20260927 07:09:44.592466 13946 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:44.588053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:40959 (local address 127.4.174.126:44821)
0927 07:09:44.588120 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:44.588122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:44.588148 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:09:44.588177 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:44.588178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:44.588190 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:44.588232 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:44.588541 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.589035 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.589739 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.589924 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.589979 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:44.592402 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:40959: BlockingRecv error: recv got EOF from 127.4.174.66:40959 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 27 07:09:44 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:44 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:44.723953 13948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:44.719457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:37145)
0927 07:09:44.719587 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:44.719600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:44.719697 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:44.719837 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:44.719838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:44.720008 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:44.720056 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:44.720058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.720705 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.720706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:44.721231 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:44.721233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.721697 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.721698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:44.721761 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:44.722172 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:44.722176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:44.723860 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
W20260927 07:09:44.724002 13946 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:44.719565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:37145 (local address 127.4.174.126:44821)
0927 07:09:44.719622 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:44.719624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:44.719656 (+ 32us) server_negotiation.cc:407] Connection header received
0927 07:09:44.719760 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:44.719761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:44.719775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:44.719821 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:44.720129 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.720626 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.721309 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:44.721568 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:44.721675 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:44.723948 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:37145: BlockingRecv error: recv got EOF from 127.4.174.67:37145 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 27 07:09:45 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:45 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:45.456225 13949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:45.451669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:52859)
0927 07:09:45.451795 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:45.451805 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:45.451853 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:45.452054 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:45.452055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:45.452216 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:45.452267 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:45.452268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.452921 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.452922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:45.453486 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:45.453488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.453899 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.453900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:45.453954 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:45.454382 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:45.454387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:45.456129 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
W20260927 07:09:45.456348 13950 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:45.451790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:52859 (local address 127.4.174.126:44821)
0927 07:09:45.451912 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:45.451914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:45.451921 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:09:45.451943 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:45.451944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:45.451953 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:45.452013 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:45.452330 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.452830 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.453554 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.453743 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.453788 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:45.456219 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:52859: BlockingRecv error: recv got EOF from 127.4.174.65:52859 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
Sep 27 07:09:45 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:45 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:45.597347 13951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:45.592889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:60133)
0927 07:09:45.593004 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:45.593015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:45.593071 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:45.593192 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:45.593192 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:45.593340 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:45.593389 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:45.593390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.594119 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.594119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:45.594712 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:45.594714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.595041 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.595042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:45.595082 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:45.595534 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:45.595539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:45.597288 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
W20260927 07:09:45.597437 13950 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:45.592982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:60133 (local address 127.4.174.126:44821)
0927 07:09:45.593043 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:45.593045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:45.593058 (+ 13us) server_negotiation.cc:407] Connection header received
0927 07:09:45.593122 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:45.593123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:45.593139 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:45.593187 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:45.593432 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.594064 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.594792 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.594984 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.595032 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:45.597377 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:60133: BlockingRecv error: recv got EOF from 127.4.174.66:60133 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 27 07:09:45 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:45 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:45.728916 13952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:45.724521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:55835)
0927 07:09:45.724689 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:45.724700 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:45.724770 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:45.724894 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:45.724894 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:45.725096 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:45.725165 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:45.725169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.725790 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.725790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:45.726312 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:45.726313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.726695 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.726695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:45.726733 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:45.727177 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:45.727181 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:45.728836 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
W20260927 07:09:45.728992 13950 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:45.724559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:55835 (local address 127.4.174.126:44821)
0927 07:09:45.724629 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:45.724631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:45.724774 (+ 143us) server_negotiation.cc:407] Connection header received
0927 07:09:45.724822 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:45.724823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:45.724837 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:45.724891 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:45.725236 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.725732 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.726378 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:45.726626 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:45.726697 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:45.728938 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:55835: BlockingRecv error: recv got EOF from 127.4.174.67:55835 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 07:09:46 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:46 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:46.461491 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:46.456748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:48393)
0927 07:09:46.456889 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:46.456901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:46.456965 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:46.457115 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:46.457116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:46.457284 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:46.457334 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:46.457336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.458139 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.458140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:46.458714 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:46.458717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.459030 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.459031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:46.459067 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:46.459536 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:46.459540 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:46.461420 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
W20260927 07:09:46.461601 13954 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:46.456856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:48393 (local address 127.4.174.126:44821)
0927 07:09:46.456965 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:46.456967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:46.457013 (+ 46us) server_negotiation.cc:407] Connection header received
0927 07:09:46.457038 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:46.457039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:46.457051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:46.457105 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:46.457400 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.458056 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.458787 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.458978 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.459030 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:46.461529 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:48393: BlockingRecv error: recv got EOF from 127.4.174.65:48393 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
Sep 27 07:09:46 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:46 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:46.602142 13955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:46.597856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:56641)
0927 07:09:46.597998 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:46.598010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:46.598080 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:46.598216 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:46.598216 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:46.598368 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:46.598419 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:46.598420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.599173 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.599174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:46.599724 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:46.599726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.600007 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.600007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:46.600042 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:46.600454 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:46.600458 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:46.602074 (+ 1616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260927 07:09:46.602227 13954 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:46.597931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:56641 (local address 127.4.174.126:44821)
0927 07:09:46.598007 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:46.598009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:46.598122 (+ 113us) server_negotiation.cc:407] Connection header received
0927 07:09:46.598150 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:46.598151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:46.598163 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:46.598205 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:46.598473 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.599101 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.599782 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.599960 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.600004 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:46.602165 (+ 2161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:56641: BlockingRecv error: recv got EOF from 127.4.174.66:56641 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 07:09:46 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:46 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:46.739059 13956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:46.729364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:35097)
0927 07:09:46.729522 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:46.729533 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:46.729602 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:46.729703 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:46.729703 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:46.729858 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:46.729905 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:46.729906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.730484 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.730485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:46.731153 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:46.731156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.731483 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.731483 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:46.731524 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:46.737142 (+ 5618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:46.737147 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:46.738984 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
W20260927 07:09:46.739161 13954 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:46.729466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:35097 (local address 127.4.174.126:44821)
0927 07:09:46.729516 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:46.729517 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:46.729583 (+ 66us) server_negotiation.cc:407] Connection header received
0927 07:09:46.729650 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:46.729651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:46.729662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:46.729697 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:46.729955 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.730432 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.731227 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:46.731428 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:46.731478 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:46.739093 (+ 7615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:35097: BlockingRecv error: recv got EOF from 127.4.174.67:35097 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 27 07:09:47 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:47 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:47.466691 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:47.461957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:47243)
0927 07:09:47.462116 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:47.462130 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:47.462196 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:47.462348 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:47.462350 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:47.462619 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:47.462680 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:47.462683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.463633 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.463634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:47.464170 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:47.464172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.464514 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.464515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:47.464580 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:47.464975 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:47.464979 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:47.466605 (+ 1626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
W20260927 07:09:47.466733 13958 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:47.462080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:47243 (local address 127.4.174.126:44821)
0927 07:09:47.462190 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:47.462192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:47.462200 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:47.462247 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:47.462247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:47.462259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:47.462318 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:47.462757 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.463554 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.464247 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.464455 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.464510 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:47.466670 (+ 2160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:47243: BlockingRecv error: recv got EOF from 127.4.174.65:47243 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
Sep 27 07:09:47 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:47 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:47.607234 13959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:47.602627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:60927)
0927 07:09:47.602769 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:47.602781 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:47.602838 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:47.602961 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:47.602962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:47.603128 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:47.603177 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:47.603179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.603861 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.603862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:47.604622 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:47.604626 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.605029 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.605030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:47.605075 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:47.605513 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:47.605517 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:47.607174 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
W20260927 07:09:47.607321 13958 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:47.602703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:60927 (local address 127.4.174.126:44821)
0927 07:09:47.602764 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:47.602766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:47.602834 (+ 68us) server_negotiation.cc:407] Connection header received
0927 07:09:47.602880 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:47.602881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:47.602892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:47.602956 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:47.603242 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.603765 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.604721 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.604970 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.605019 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:47.607261 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:60927: BlockingRecv error: recv got EOF from 127.4.174.66:60927 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 27 07:09:47 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:47 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:47.744274 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:47.739558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:40043)
0927 07:09:47.739710 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:47.739723 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:47.739788 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:47.739951 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:47.739952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:47.740131 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:47.740182 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:47.740183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.740898 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.740899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:47.741439 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:47.741441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.741806 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.741807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:47.741866 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:47.742355 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:47.742359 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:47.744202 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":61,"threads_started":1}
W20260927 07:09:47.744370 13958 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:47.739684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:40043 (local address 127.4.174.126:44821)
0927 07:09:47.739751 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:47.739753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:47.739767 (+ 14us) server_negotiation.cc:407] Connection header received
0927 07:09:47.739868 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:47.739869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:47.739883 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:47.739927 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:47.740258 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.740826 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.741531 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:47.741739 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:47.741800 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:47.744301 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:40043: BlockingRecv error: recv got EOF from 127.4.174.67:40043 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 27 07:09:48 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:48 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:48.471661 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:48.467210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:37017)
0927 07:09:48.467342 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:48.467351 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:48.467397 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:48.467560 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:48.467561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:48.467711 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:48.467757 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:48.467758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.468382 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.468382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:48.468916 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:48.468918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.469221 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.469221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:48.469259 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:48.469680 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:48.469685 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:48.471576 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
W20260927 07:09:48.471766 13962 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:48.467319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:37017 (local address 127.4.174.126:44821)
0927 07:09:48.467432 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:48.467435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:48.467445 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:48.467470 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:48.467471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:48.467483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:48.467548 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:48.467816 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.468323 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.468983 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.469168 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.469214 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:48.471699 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:37017: BlockingRecv error: recv got EOF from 127.4.174.65:37017 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
Sep 27 07:09:48 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:48 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:48.612367 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:48.607778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:50671)
0927 07:09:48.607899 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:48.607910 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:48.607979 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:48.608140 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:48.608141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:48.608322 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:48.608366 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:48.608368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.609037 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.609038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:48.609576 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:48.609577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.609918 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.609919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:48.609975 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:48.610477 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:48.610482 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:48.612279 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
W20260927 07:09:48.612438 13962 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:48.607877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:50671 (local address 127.4.174.126:44821)
0927 07:09:48.607956 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:48.607958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:48.607969 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:09:48.608062 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:48.608063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:48.608074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:48.608123 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:48.608443 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.608955 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.609658 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.609856 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.609912 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:48.612373 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:50671: BlockingRecv error: recv got EOF from 127.4.174.66:50671 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 27 07:09:48 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:48 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:48.749153 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:48.744707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:33191)
0927 07:09:48.744853 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:48.744866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:48.744940 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:48.745076 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:48.745077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:48.745254 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:48.745297 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:48.745299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.745946 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.745947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:48.746490 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:48.746492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.746841 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.746842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:48.746897 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:48.747338 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:48.747343 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:48.749083 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
W20260927 07:09:48.749246 13962 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:48.744796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:33191 (local address 127.4.174.126:44821)
0927 07:09:48.744857 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:48.744860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:48.744931 (+ 71us) server_negotiation.cc:407] Connection header received
0927 07:09:48.745004 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:48.745005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:48.745018 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:48.745058 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:48.745375 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.745871 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.746593 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:48.746778 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:48.746833 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:48.749179 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:33191: BlockingRecv error: recv got EOF from 127.4.174.67:33191 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 27 07:09:49 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:49 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:49.476413 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:49.472123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:60367)
0927 07:09:49.472249 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:49.472260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:49.472314 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:49.472464 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:49.472465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:49.472642 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:49.472687 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:49.472689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.473313 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.473313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:49.473848 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:49.473850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.474189 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.474189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:49.474242 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:49.474693 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:49.474697 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:49.476332 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
W20260927 07:09:49.476521 13966 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:49.472228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:60367 (local address 127.4.174.126:44821)
0927 07:09:49.472347 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:49.472349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:49.472357 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:09:49.472380 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:49.472380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:49.472389 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:49.472455 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:49.472746 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.473258 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.473915 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.474123 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.474174 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:49.476450 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:60367: BlockingRecv error: recv got EOF from 127.4.174.65:60367 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":54,"threads_started":1}
Sep 27 07:09:49 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:49 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:49.617405 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:49.612814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:39203)
0927 07:09:49.612947 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:49.612960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:49.613030 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:49.613178 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:49.613179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:49.613349 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:49.613394 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:49.613396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.614053 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.614054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:49.614694 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:49.614697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.615031 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.615032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:49.615085 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:49.615509 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:49.615513 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:49.617318 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
W20260927 07:09:49.617470 13966 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:49.612901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:39203 (local address 127.4.174.126:44821)
0927 07:09:49.612985 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:49.612987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:49.613011 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:09:49.613090 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:49.613091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:49.613102 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:49.613147 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:49.613468 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.613977 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.614772 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.614972 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.615028 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:49.617418 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:39203: BlockingRecv error: recv got EOF from 127.4.174.66:39203 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 07:09:49 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:49 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:49.758915 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:49.749643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:39887)
0927 07:09:49.749756 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:49.749768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:49.749834 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:49.749976 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:49.749977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:49.750146 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:49.750189 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:49.750190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.750890 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.750891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:49.751689 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:49.751693 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.756630 (+ 4937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.756632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:49.756710 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:49.757245 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:49.757251 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:49.758846 (+ 1595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
W20260927 07:09:49.759006 13966 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:49.749732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:39887 (local address 127.4.174.126:44821)
0927 07:09:49.749777 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:49.749779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:49.749835 (+ 56us) server_negotiation.cc:407] Connection header received
0927 07:09:49.749893 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:49.749894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:49.749905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:49.749948 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:49.750263 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.750811 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.756183 (+ 5372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:49.756487 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:49.756555 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:49.758945 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:39887: BlockingRecv error: recv got EOF from 127.4.174.67:39887 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 27 07:09:50 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:50 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:50.481421 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:50.476895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:37261)
0927 07:09:50.477020 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:50.477034 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:50.477117 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:50.477283 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:50.477284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:50.477441 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:50.477490 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:50.477492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.478351 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.478352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:50.479004 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:50.479007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.479336 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.479337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:50.479376 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:50.479758 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:50.479762 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:50.481355 (+ 1593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
W20260927 07:09:50.481508 13970 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:50.476965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:37261 (local address 127.4.174.126:44821)
0927 07:09:50.477127 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:50.477130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:50.477140 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:50.477169 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:50.477169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:50.477182 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:50.477241 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:50.477557 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.478301 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.479068 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.479282 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.479332 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:50.481441 (+ 2109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:37261: BlockingRecv error: recv got EOF from 127.4.174.65:37261 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
Sep 27 07:09:50 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:50 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:50.622347 13971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:50.617925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:42611)
0927 07:09:50.618075 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:50.618085 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:50.618134 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:50.618257 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:50.618257 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:50.618441 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:50.618519 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:50.618522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.619303 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.619304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:50.619820 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:50.619821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.620153 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.620154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:50.620204 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:50.620616 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:50.620620 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:50.622269 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
W20260927 07:09:50.622417 13970 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:50.618060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:42611 (local address 127.4.174.126:44821)
0927 07:09:50.618115 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:50.618117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:50.618126 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:50.618180 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:50.618181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:50.618190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:50.618242 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:50.618611 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.619239 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.619883 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.620067 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.620114 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:50.622366 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:42611: BlockingRecv error: recv got EOF from 127.4.174.66:42611 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 27 07:09:50 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:50 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:50.763885 13972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:50.759452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:41859)
0927 07:09:50.759611 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:50.759622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:50.759688 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:50.759836 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:50.759837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:50.760048 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:50.760099 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:50.760101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.760729 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.760730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:50.761280 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:50.761282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.761574 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.761574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:50.761616 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:50.762080 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:50.762085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:50.763816 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
W20260927 07:09:50.763950 13970 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:50.759510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:41859 (local address 127.4.174.126:44821)
0927 07:09:50.759574 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:50.759576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:50.759683 (+ 107us) server_negotiation.cc:407] Connection header received
0927 07:09:50.759746 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:50.759747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:50.759762 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:50.759816 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:50.760161 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.760646 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.761345 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:50.761516 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:50.761573 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:50.763900 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:41859: BlockingRecv error: recv got EOF from 127.4.174.67:41859 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 27 07:09:51 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:51 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:51.486218 13973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:51.481900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:49567)
0927 07:09:51.482018 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:51.482044 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:51.482107 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:51.482232 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:51.482232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:51.482387 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:51.482436 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:51.482438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.483070 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.483071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:51.483585 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:51.483586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.483898 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.483899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:51.483937 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:51.484325 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:51.484329 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:51.486136 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
W20260927 07:09:51.486325 13974 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:51.481965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:49567 (local address 127.4.174.126:44821)
0927 07:09:51.482110 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:51.482111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:51.482140 (+ 29us) server_negotiation.cc:407] Connection header received
0927 07:09:51.482161 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:51.482161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:51.482172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:51.482220 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:51.482489 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.483019 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.483648 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.483842 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.483892 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:51.486256 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:49567: BlockingRecv error: recv got EOF from 127.4.174.65:49567 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
Sep 27 07:09:51 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:51 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:51.627442 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:51.622896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:38003)
0927 07:09:51.623033 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:51.623044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:51.623112 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:51.623276 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:51.623277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:51.623441 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:51.623491 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:51.623493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.624178 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.624180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:51.624792 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:51.624795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.625179 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.625180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:51.625240 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:51.625643 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:51.625647 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:51.627380 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
W20260927 07:09:51.627529 13974 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:51.623012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:38003 (local address 127.4.174.126:44821)
0927 07:09:51.623058 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:51.623060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:51.623093 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:09:51.623165 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:51.623166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:51.623176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:51.623231 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:51.623572 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.624070 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.624883 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.625093 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.625144 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:51.627468 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:38003: BlockingRecv error: recv got EOF from 127.4.174.66:38003 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 27 07:09:51 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:51 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.67@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:51.773886 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:51.764363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.67:50177)
0927 07:09:51.764543 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:51.764555 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:51.764614 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:51.764718 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:51.764718 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:51.764888 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:51.764935 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:51.764937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.765546 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.765547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:51.766068 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:51.766069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.766379 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.766379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:51.766414 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:51.771851 (+ 5437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:51.771856 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:51.773824 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":61,"threads_started":1}
W20260927 07:09:51.774010 13974 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:51.764386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:50177 (local address 127.4.174.126:44821)
0927 07:09:51.764453 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:51.764455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:51.764611 (+ 156us) server_negotiation.cc:407] Connection header received
0927 07:09:51.764665 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:51.764666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:51.764678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:51.764713 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:51.764984 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.765490 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.766137 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:51.766322 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:51.766373 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:51.773946 (+ 7573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.67:50177: BlockingRecv error: recv got EOF from 127.4.174.67:50177 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 27 07:09:52 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:52 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.65@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:52.491616 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:52.486843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.65:47093)
0927 07:09:52.486964 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:52.486974 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:52.487036 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:52.487189 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:52.487190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:52.487341 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:52.487393 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:52.487394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.488066 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.488067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:52.488649 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:52.488651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.489064 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.489065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:52.489127 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:52.489579 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:52.489583 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:52.491543 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
W20260927 07:09:52.491698 13978 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:52.486922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:47093 (local address 127.4.174.126:44821)
0927 07:09:52.487057 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:52.487060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:52.487071 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:09:52.487097 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:52.487098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:52.487108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:52.487172 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:52.487442 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.487994 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.488733 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.488984 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.489056 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:52.491634 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.65:47093: BlockingRecv error: recv got EOF from 127.4.174.65:47093 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
Sep 27 07:09:52 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:09:52 dist-test-slave-zw7p krb5kdc[12920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.174.66@KRBTEST.COM for oryx/127.4.174.126@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:09:52.632637 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:52.627997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44821 (local address 127.4.174.66:46027)
0927 07:09:52.628144 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:52.628155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:52.628206 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:52.628362 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:52.628362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:52.628523 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:52.628573 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:52.628575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.629227 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.629228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:52.629872 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:52.629877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.630291 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.630292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:52.630335 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:52.630832 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:52.630836 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:52.632546 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.174.126:44821: Server oryx/127.4.174.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
W20260927 07:09:52.632700 13978 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:52.628126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:46027 (local address 127.4.174.126:44821)
0927 07:09:52.628223 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:52.628225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:52.628235 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:09:52.628263 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:52.628264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:52.628275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:52.628358 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:52.628633 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.629143 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.629955 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.630229 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.630286 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:52.632639 (+ 2353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.174.66:46027: BlockingRecv error: recv got EOF from 127.4.174.66:46027 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 07:09:52.672574 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 13504
I20260927 07:09:52.676604 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 13655
I20260927 07:09:52.680330 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 13788
I20260927 07:09:52.683822 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 13433
2026-09-27T07:09:52Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17304 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-27T07:09:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:52Z Disabled control of system clock
I20260927 07:09:52.709069 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:38537
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:38537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 07:09:52.786132 13983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:52.786330 13983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:52.786348 13983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:52.787554 13983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:52.787593 13983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:52.787606 13983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:52.787618 13983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:52.787642 13983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:52.789031 13983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:38537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:38537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:52.789259 13983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:52.789526 13983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 07:09:52.794212 13983 server_base.cc:1034] running on GCE node
W20260927 07:09:52.794282 13989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:52.794312 13988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:52.794275 13991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:52.794780 13983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:52.795066 13983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:52.796235 13983 hybrid_clock.cc:648] HybridClock initialized: now 1790492992796219 us; error 26 us; skew 500 ppm
I20260927 07:09:52.797343 13983 webserver.cc:460] Webserver started at http://127.4.174.126:39385/ using document root <none> and password file <none>
I20260927 07:09:52.797551 13983 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:52.797598 13983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:52.797719 13983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:52.798792 13983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "7ef584c1e830497ba3d64b2f8e7f30fd"
format_stamp: "Formatted at 2026-09-27 07:09:52 on dist-test-slave-zw7p"
server_key: "cd4f0d024b8cb71ef6563afc77a86191"
server_key_iv: "469b5ea50e1ac777c72d144a0f3d631a"
server_key_version: "encryptionkey@0"
I20260927 07:09:52.799135 13983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "7ef584c1e830497ba3d64b2f8e7f30fd"
format_stamp: "Formatted at 2026-09-27 07:09:52 on dist-test-slave-zw7p"
server_key: "cd4f0d024b8cb71ef6563afc77a86191"
server_key_iv: "469b5ea50e1ac777c72d144a0f3d631a"
server_key_version: "encryptionkey@0"
I20260927 07:09:52.800330 13983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:52.800981 13997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:52.801136 13983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:52.801216 13983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "7ef584c1e830497ba3d64b2f8e7f30fd"
format_stamp: "Formatted at 2026-09-27 07:09:52 on dist-test-slave-zw7p"
server_key: "cd4f0d024b8cb71ef6563afc77a86191"
server_key_iv: "469b5ea50e1ac777c72d144a0f3d631a"
server_key_version: "encryptionkey@0"
I20260927 07:09:52.801272 13983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:52.824424 13983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:52.824748 13983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:52.824880 13983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:52.828488 13983 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:38537
I20260927 07:09:52.828545 14049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:38537 every 8 connection(s)
I20260927 07:09:52.828862 13983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:52.829563 14050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:52.832177 14050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd: Bootstrap starting.
I20260927 07:09:52.832748 14050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:52.832996 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 13983
I20260927 07:09:52.833027 14050 log.cc:824] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:52.833077 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:52.833288 4793 external_mini_cluster.cc:1428] Setting key e765272861a69d34dc7c10d65d824bbb
I20260927 07:09:52.833742 14050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd: No bootstrap required, opened a new log
I20260927 07:09:52.835812 14050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef584c1e830497ba3d64b2f8e7f30fd" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38537 } }
I20260927 07:09:52.835975 14050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:52.836011 14050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef584c1e830497ba3d64b2f8e7f30fd, State: Initialized, Role: FOLLOWER
I20260927 07:09:52.836143 14050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef584c1e830497ba3d64b2f8e7f30fd" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38537 } }
I20260927 07:09:52.836239 14050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:52.836274 14050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:52.836335 14050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:52.836922 14050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef584c1e830497ba3d64b2f8e7f30fd" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38537 } }
I20260927 07:09:52.837042 14050 leader_election.cc:302] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ef584c1e830497ba3d64b2f8e7f30fd; no voters:
I20260927 07:09:52.837208 14050 leader_election.cc:288] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:52.837245 14055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:52.837411 14055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [term 1 LEADER]: Becoming Leader. State: Replica: 7ef584c1e830497ba3d64b2f8e7f30fd, State: Running, Role: LEADER
I20260927 07:09:52.837456 14050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:52.837519 14055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef584c1e830497ba3d64b2f8e7f30fd" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38537 } }
I20260927 07:09:52.837831 14055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ef584c1e830497ba3d64b2f8e7f30fd. Latest consensus state: current_term: 1 leader_uuid: "7ef584c1e830497ba3d64b2f8e7f30fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef584c1e830497ba3d64b2f8e7f30fd" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38537 } } }
I20260927 07:09:52.837983 14055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:52.838219 14063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:52.838757 14063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:52.839157 14056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ef584c1e830497ba3d64b2f8e7f30fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef584c1e830497ba3d64b2f8e7f30fd" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38537 } } }
I20260927 07:09:52.839635 14056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:52.839964 14054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:52.833760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51868 (local address 127.4.174.126:38537)
0927 07:09:52.833928 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:52.833936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:52.833960 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:09:52.834892 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:52.834900 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:52.834962 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:52.835082 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:52.835334 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.836674 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.837729 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:52.838320 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:52.838473 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:52.839176 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:52.839180 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:52.839180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:52.839390 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:52.839429 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:52.839440 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:52.839503 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:52.839692 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 07:09:52.839774 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260927 07:09:52.840874 14063 catalog_manager.cc:1347] Generated new cluster ID: 0551e774ce09479d87ee0c62414c6cc6
I20260927 07:09:52.840924 14063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:52.858343 14063 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:52.859110 14063 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:52.868402 14063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ef584c1e830497ba3d64b2f8e7f30fd: Generated new TSK 0
I20260927 07:09:52.868644 14063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:09:52.878871 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38537
--builtin_ntp_servers=127.4.174.84:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 07:09:52.965116 14074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:52.965320 14074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:52.965349 14074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:52.966493 14074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:52.966529 14074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:52.966585 14074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:52.967880 14074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:52.968098 14074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:52.968374 14074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:52.973176 14080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:52.973325 14074 server_base.cc:1034] running on GCE node
W20260927 07:09:52.973178 14079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:52.973173 14082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:52.973738 14074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:52.973982 14074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:52.975124 14074 hybrid_clock.cc:648] HybridClock initialized: now 1790492992975110 us; error 30 us; skew 500 ppm
I20260927 07:09:52.976250 14074 webserver.cc:460] Webserver started at http://127.4.174.65:42521/ using document root <none> and password file <none>
I20260927 07:09:52.976459 14074 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:52.976508 14074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:52.976630 14074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:52.977654 14074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "02aacec888bd4f75b03aab9586002abe"
format_stamp: "Formatted at 2026-09-27 07:09:52 on dist-test-slave-zw7p"
server_key: "9e8a40f99b3b6d0f8ecebf1bf7b80efc"
server_key_iv: "eeff0532cac71b0091994934dd8ecb7f"
server_key_version: "encryptionkey@0"
I20260927 07:09:52.977974 14074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "02aacec888bd4f75b03aab9586002abe"
format_stamp: "Formatted at 2026-09-27 07:09:52 on dist-test-slave-zw7p"
server_key: "9e8a40f99b3b6d0f8ecebf1bf7b80efc"
server_key_iv: "eeff0532cac71b0091994934dd8ecb7f"
server_key_version: "encryptionkey@0"
I20260927 07:09:52.979218 14074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:52.979846 14088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:52.980000 14074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:52.980078 14074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "02aacec888bd4f75b03aab9586002abe"
format_stamp: "Formatted at 2026-09-27 07:09:52 on dist-test-slave-zw7p"
server_key: "9e8a40f99b3b6d0f8ecebf1bf7b80efc"
server_key_iv: "eeff0532cac71b0091994934dd8ecb7f"
server_key_version: "encryptionkey@0"
I20260927 07:09:52.980134 14074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:53.006742 14074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:53.007057 14074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:53.007187 14074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:53.007413 14074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:53.007726 14074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:53.007761 14074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.007789 14074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:53.007812 14074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.013165 14074 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:34477
I20260927 07:09:53.013209 14201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:34477 every 8 connection(s)
I20260927 07:09:53.013531 14074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:53.018476 14054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.014207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:39011 (local address 127.4.174.126:38537)
0927 07:09:53.014254 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.014256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.014484 (+ 228us) server_negotiation.cc:407] Connection header received
0927 07:09:53.015024 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.015025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.015041 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.015101 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.015736 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.016224 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.017148 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.017349 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.017417 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:53.018164 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:53.018165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:53.018166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:53.018171 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:53.018186 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:53.018190 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:53.018239 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:53.018365 (+ 126us) server_negotiation.cc:299] Negotiation successful
0927 07:09:53.018392 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:53.018586 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.014194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38537 (local address 127.4.174.65:39011)
0927 07:09:53.014347 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:53.014381 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:53.014901 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:53.015181 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:53.015185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:53.015194 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:53.015674 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:53.015678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.016282 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.016282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:53.017053 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:53.017055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.017443 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.017444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:53.017547 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:53.018015 (+ 468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:53.018015 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:53.018044 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:53.018066 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:53.018245 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:53.018248 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:53.018251 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:53.018299 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 07:09:53.018379 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260927 07:09:53.019129 14202 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38537
I20260927 07:09:53.019214 14202 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:53.019361 14202 heartbeater.cc:511] Master 127.4.174.126:38537 requested a full tablet report, sending...
I20260927 07:09:53.019856 14014 ts_manager.cc:194] Registered new tserver with Master: 02aacec888bd4f75b03aab9586002abe (127.4.174.65:34477)
I20260927 07:09:53.020460 14014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.65:39011
I20260927 07:09:53.023753 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 14074
I20260927 07:09:53.023835 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:53.024017 4793 external_mini_cluster.cc:1428] Setting key b4a06ad3b1114725a4e49531dd9224d6
I20260927 07:09:53.024838 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38537
--builtin_ntp_servers=127.4.174.84:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 07:09:53.102727 14205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:53.102934 14205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:53.102962 14205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:53.104130 14205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:53.104168 14205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:53.104202 14205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:53.105521 14205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:53.105749 14205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:53.106027 14205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:53.110791 14213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.110777 14210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.110795 14211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:53.111088 14205 server_base.cc:1034] running on GCE node
I20260927 07:09:53.111281 14205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:53.111487 14205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:53.112617 14205 hybrid_clock.cc:648] HybridClock initialized: now 1790492993112603 us; error 29 us; skew 500 ppm
I20260927 07:09:53.113788 14205 webserver.cc:460] Webserver started at http://127.4.174.66:33675/ using document root <none> and password file <none>
I20260927 07:09:53.114037 14205 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:53.114087 14205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:53.114223 14205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:53.115289 14205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "6203f0a3ec244883b2e7b18461b818e7"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "967787d726312246a5db72c2520e65e7"
server_key_iv: "f8fe5f38a688ba0f902a7d8739694b6f"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.115604 14205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "6203f0a3ec244883b2e7b18461b818e7"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "967787d726312246a5db72c2520e65e7"
server_key_iv: "f8fe5f38a688ba0f902a7d8739694b6f"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.116758 14205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:53.117405 14219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.117553 14205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:53.117626 14205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "6203f0a3ec244883b2e7b18461b818e7"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "967787d726312246a5db72c2520e65e7"
server_key_iv: "f8fe5f38a688ba0f902a7d8739694b6f"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.117686 14205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:53.140435 14205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:53.140755 14205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:53.140890 14205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:53.141119 14205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:53.141445 14205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:53.141480 14205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.141511 14205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:53.141534 14205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.146755 14205 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:41955
I20260927 07:09:53.146806 14332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:41955 every 8 connection(s)
I20260927 07:09:53.147133 14205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:53.149125 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 14205
I20260927 07:09:53.149204 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:53.149398 4793 external_mini_cluster.cc:1428] Setting key bc5dadfd0c1b086c8ff158e878244fcd
I20260927 07:09:53.150247 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38537
--builtin_ntp_servers=127.4.174.84:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 07:09:53.152508 14054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.147825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:57907 (local address 127.4.174.126:38537)
0927 07:09:53.147889 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.147891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.148127 (+ 236us) server_negotiation.cc:407] Connection header received
0927 07:09:53.148632 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.148632 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.148642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.148691 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.149361 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.149920 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.151056 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.151325 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.151398 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:53.152255 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:53.152256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:53.152257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:53.152261 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:53.152275 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:53.152278 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:53.152307 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:53.152394 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 07:09:53.152428 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:09:53.152701 14335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.147827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38537 (local address 127.4.174.66:57907)
0927 07:09:53.148006 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:53.148040 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:53.148551 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:53.148747 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:53.148750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:53.148758 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:53.149261 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:53.149269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.150013 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.150014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:53.150981 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:53.150984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.151418 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.151419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:53.151584 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:53.152097 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:53.152098 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:53.152137 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:53.152166 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:53.152320 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:53.152325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:53.152328 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:53.152391 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 07:09:53.152492 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260927 07:09:53.153146 14333 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38537
I20260927 07:09:53.153228 14333 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:53.153401 14333 heartbeater.cc:511] Master 127.4.174.126:38537 requested a full tablet report, sending...
I20260927 07:09:53.153779 14014 ts_manager.cc:194] Registered new tserver with Master: 6203f0a3ec244883b2e7b18461b818e7 (127.4.174.66:41955)
I20260927 07:09:53.154259 14014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.66:57907
W20260927 07:09:53.230397 14336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:53.230623 14336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:53.230643 14336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:53.231752 14336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:53.231786 14336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:53.231809 14336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:53.233104 14336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:38537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:53.233291 14336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:53.233536 14336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:53.238166 14344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:53.238226 14336 server_base.cc:1034] running on GCE node
W20260927 07:09:53.238170 14341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.238161 14342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:53.238629 14336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:53.238925 14336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:53.240078 14336 hybrid_clock.cc:648] HybridClock initialized: now 1790492993240061 us; error 31 us; skew 500 ppm
I20260927 07:09:53.241255 14336 webserver.cc:460] Webserver started at http://127.4.174.67:37117/ using document root <none> and password file <none>
I20260927 07:09:53.241482 14336 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:53.241526 14336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:53.241650 14336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:53.242761 14336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "2e1a51650b3c4d94a5527e8edd468cda"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "db4827790de86b161f71e6f44a6c1371"
server_key_iv: "ac86b0fa740046450b183fd02480a8f8"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.243088 14336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "2e1a51650b3c4d94a5527e8edd468cda"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "db4827790de86b161f71e6f44a6c1371"
server_key_iv: "ac86b0fa740046450b183fd02480a8f8"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.244294 14336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:53.245002 14350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.245142 14336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:53.245222 14336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "2e1a51650b3c4d94a5527e8edd468cda"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "db4827790de86b161f71e6f44a6c1371"
server_key_iv: "ac86b0fa740046450b183fd02480a8f8"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.245287 14336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:53.264099 14336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:53.264396 14336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:53.264526 14336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:53.264742 14336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:53.265079 14336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:53.265112 14336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.265142 14336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:53.265165 14336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.270468 14336 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:41721
I20260927 07:09:53.270530 14463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:41721 every 8 connection(s)
I20260927 07:09:53.270869 14336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:53.275293 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 14336
I20260927 07:09:53.275379 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:53.275570 4793 external_mini_cluster.cc:1428] Setting key f1620d5327c2413c355bccde6046395b
I20260927 07:09:53.275900 14054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.271636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:46373 (local address 127.4.174.126:38537)
0927 07:09:53.271703 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.271705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.271893 (+ 188us) server_negotiation.cc:407] Connection header received
0927 07:09:53.272393 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.272395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.272410 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.272491 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.273121 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.273623 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.274547 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.274775 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.274843 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:53.275650 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:53.275651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:53.275651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:53.275656 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:53.275672 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:53.275676 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:53.275721 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:53.275794 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 07:09:53.275820 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:09:53.276108 14466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.271574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38537 (local address 127.4.174.67:46373)
0927 07:09:53.271752 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:53.271793 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:53.272280 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:53.272549 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:53.272552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:53.272560 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:53.273038 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:53.273042 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.273693 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.273694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:53.274462 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:53.274464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:53.274858 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:53.274859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:53.274973 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:53.275515 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:53.275516 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:53.275544 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:53.275567 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:53.275727 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:53.275731 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:53.275735 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:09:53.275797 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 07:09:53.275898 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260927 07:09:53.276479 14464 heartbeater.cc:347] Connected to a master server at 127.4.174.126:38537
I20260927 07:09:53.276567 14464 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:53.276727 14464 heartbeater.cc:511] Master 127.4.174.126:38537 requested a full tablet report, sending...
I20260927 07:09:53.277062 14014 ts_manager.cc:194] Registered new tserver with Master: 2e1a51650b3c4d94a5527e8edd468cda (127.4.174.67:41721)
I20260927 07:09:53.277428 14014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.67:46373
I20260927 07:09:53.287286 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 07:09:53.289037 14054 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:53.288528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.4.174.126:38537)
0927 07:09:53.288577 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.288579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.288697 (+ 118us) server_negotiation.cc:407] Connection header received
0927 07:09:53.288760 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.288761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.288771 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.288815 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.288968 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51876: BlockingRecv error: recv got EOF from 127.0.0.1:51876 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 07:09:53.289083 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 14074
I20260927 07:09:53.293196 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 14205
I20260927 07:09:53.297107 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 14336
I20260927 07:09:53.300691 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 13983
2026-09-27T07:09:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (615 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-27T07:09:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:53Z Disabled control of system clock
I20260927 07:09:53.325183 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:33485
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:33485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 07:09:53.402323 14475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:53.402542 14475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:53.402590 14475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:53.403741 14475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:53.403784 14475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:53.403800 14475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:53.403816 14475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:53.403831 14475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:53.405148 14475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:33485
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:33485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:53.405385 14475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:53.405649 14475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:53.410182 14480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.410184 14481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.410207 14483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:53.410598 14475 server_base.cc:1034] running on GCE node
I20260927 07:09:53.410842 14475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:53.411118 14475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:53.412254 14475 hybrid_clock.cc:648] HybridClock initialized: now 1790492993412241 us; error 31 us; skew 500 ppm
I20260927 07:09:53.413452 14475 webserver.cc:460] Webserver started at http://127.4.174.126:44379/ using document root <none> and password file <none>
I20260927 07:09:53.413635 14475 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:53.413672 14475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:53.413769 14475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:53.414803 14475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "2b009d8fe78044efa54ed12d7b81b4bb"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "09e3d0abc7d681020b57157b7854eabb"
server_key_iv: "fa26367e045caf617e84f12c3d32a099"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.415091 14475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "2b009d8fe78044efa54ed12d7b81b4bb"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "09e3d0abc7d681020b57157b7854eabb"
server_key_iv: "fa26367e045caf617e84f12c3d32a099"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.416190 14475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:53.416808 14489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.416985 14475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:53.417068 14475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "2b009d8fe78044efa54ed12d7b81b4bb"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "09e3d0abc7d681020b57157b7854eabb"
server_key_iv: "fa26367e045caf617e84f12c3d32a099"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.417136 14475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:53.420320 14475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:53.420554 14475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:53.420665 14475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:53.424057 14475 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:33485
I20260927 07:09:53.424085 14541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:33485 every 8 connection(s)
I20260927 07:09:53.424363 14475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:53.424962 14542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:53.427572 14542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb: Bootstrap starting.
I20260927 07:09:53.428097 14542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:53.428349 14542 log.cc:824] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:53.428723 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 14475
I20260927 07:09:53.428812 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:53.429013 4793 external_mini_cluster.cc:1428] Setting key 23c9fa81edfcab28217d3f51527ec091
I20260927 07:09:53.429036 14542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb: No bootstrap required, opened a new log
I20260927 07:09:53.431103 14542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b009d8fe78044efa54ed12d7b81b4bb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33485 } }
I20260927 07:09:53.431258 14542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:53.431295 14542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b009d8fe78044efa54ed12d7b81b4bb, State: Initialized, Role: FOLLOWER
I20260927 07:09:53.431461 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.429501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58266 (local address 127.4.174.126:33485)
0927 07:09:53.429675 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.429683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.429706 (+ 23us) server_negotiation.cc:407] Connection header received
0927 07:09:53.430224 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.430230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.430267 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.430375 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.431032 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:53.431036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:53.431036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:53.431039 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:53.431076 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:53.431085 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:53.431136 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:53.431216 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 07:09:53.431266 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260927 07:09:53.431444 14542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b009d8fe78044efa54ed12d7b81b4bb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33485 } }
I20260927 07:09:53.431536 14542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:53.431571 14542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:53.431614 14542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:53.432394 14542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b009d8fe78044efa54ed12d7b81b4bb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33485 } }
I20260927 07:09:53.432495 14542 leader_election.cc:302] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b009d8fe78044efa54ed12d7b81b4bb; no voters:
I20260927 07:09:53.432641 14542 leader_election.cc:288] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:53.432675 14547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:53.432799 14547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [term 1 LEADER]: Becoming Leader. State: Replica: 2b009d8fe78044efa54ed12d7b81b4bb, State: Running, Role: LEADER
I20260927 07:09:53.432873 14542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:53.432899 14547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b009d8fe78044efa54ed12d7b81b4bb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33485 } }
I20260927 07:09:53.433211 14547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b009d8fe78044efa54ed12d7b81b4bb. Latest consensus state: current_term: 1 leader_uuid: "2b009d8fe78044efa54ed12d7b81b4bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b009d8fe78044efa54ed12d7b81b4bb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33485 } } }
I20260927 07:09:53.433285 14547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:53.433243 14549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b009d8fe78044efa54ed12d7b81b4bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b009d8fe78044efa54ed12d7b81b4bb" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 33485 } } }
I20260927 07:09:53.433374 14549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb [sys.catalog]: This master's current role is: LEADER
W20260927 07:09:53.434475 14563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:09:53.434516 14563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:09:53.434585 14553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:53.434741 14553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:53.436065 14553 catalog_manager.cc:1347] Generated new cluster ID: f852b2c8725d436bbc39e5b695115048
I20260927 07:09:53.436115 14553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:09:53.472054 14553 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:53.472237 14553 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:53.480288 14553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b009d8fe78044efa54ed12d7b81b4bb: Generated new TSK 0
I20260927 07:09:53.480479 14553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:09:53.491722 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33485
--builtin_ntp_servers=127.4.174.84:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 07:09:53.570266 14566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:53.570474 14566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:53.570497 14566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:53.571645 14566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:53.571682 14566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:53.571707 14566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:53.573001 14566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:53.573220 14566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:53.573493 14566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:53.578141 14571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.578141 14574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:53.578171 14566 server_base.cc:1034] running on GCE node
W20260927 07:09:53.578168 14572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:53.578507 14566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:53.578768 14566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:53.579942 14566 hybrid_clock.cc:648] HybridClock initialized: now 1790492993579927 us; error 28 us; skew 500 ppm
I20260927 07:09:53.581105 14566 webserver.cc:460] Webserver started at http://127.4.174.65:39849/ using document root <none> and password file <none>
I20260927 07:09:53.581319 14566 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:53.581368 14566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:53.581496 14566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:53.582507 14566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "dae99af5c4884fd6a3ef46c3d7c5ed68"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "efc9f42c6230ff95fab3885641f9f7b2"
server_key_iv: "3efe4dbd54b152a7ac77e9faf2181abb"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.582880 14566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "dae99af5c4884fd6a3ef46c3d7c5ed68"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "efc9f42c6230ff95fab3885641f9f7b2"
server_key_iv: "3efe4dbd54b152a7ac77e9faf2181abb"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.584125 14566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:53.584765 14580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.584914 14566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:53.585004 14566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "dae99af5c4884fd6a3ef46c3d7c5ed68"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "efc9f42c6230ff95fab3885641f9f7b2"
server_key_iv: "3efe4dbd54b152a7ac77e9faf2181abb"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.585071 14566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:53.588331 14566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:53.588567 14566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:53.588685 14566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:53.588864 14566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:53.589143 14566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:53.589175 14566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.589200 14566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:53.589226 14566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.594458 14566 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:37979
I20260927 07:09:53.594521 14693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:37979 every 8 connection(s)
I20260927 07:09:53.594812 14566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:09:53.596237 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 14566
I20260927 07:09:53.596318 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:53.596508 4793 external_mini_cluster.cc:1428] Setting key c5e3de06481ad5bfd099a27c6bd3dd98
I20260927 07:09:53.597409 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33485
--builtin_ntp_servers=127.4.174.84:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 07:09:53.597762 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.595449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:35157 (local address 127.4.174.126:33485)
0927 07:09:53.595499 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.595501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.595729 (+ 228us) server_negotiation.cc:407] Connection header received
0927 07:09:53.596224 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.596226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.596233 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.596284 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.597520 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:53.597522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:53.597522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:53.597528 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:53.597544 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:53.597549 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:53.597578 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:53.597679 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 07:09:53.597694 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:09:53.597950 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.595434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33485 (local address 127.4.174.65:35157)
0927 07:09:53.595609 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:53.595649 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:53.596115 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:53.596356 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:53.596360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:53.596370 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:53.597361 (+ 991us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:53.597361 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:53.597390 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:53.597421 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:53.597613 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:53.597616 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:53.597618 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:09:53.597679 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 07:09:53.597740 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260927 07:09:53.598352 14694 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33485
I20260927 07:09:53.598449 14694 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:53.598675 14694 heartbeater.cc:511] Master 127.4.174.126:33485 requested a full tablet report, sending...
I20260927 07:09:53.599232 14506 ts_manager.cc:194] Registered new tserver with Master: dae99af5c4884fd6a3ef46c3d7c5ed68 (127.4.174.65:37979)
W20260927 07:09:53.686610 14697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:53.686856 14697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:53.686885 14697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:53.688153 14697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:53.688192 14697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:53.688218 14697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:53.689585 14697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:53.689819 14697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:53.690100 14697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:53.694818 14705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.694937 14703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.694970 14702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:53.695014 14697 server_base.cc:1034] running on GCE node
I20260927 07:09:53.695263 14697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:53.695463 14697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:53.696620 14697 hybrid_clock.cc:648] HybridClock initialized: now 1790492993696606 us; error 25 us; skew 500 ppm
I20260927 07:09:53.697739 14697 webserver.cc:460] Webserver started at http://127.4.174.66:36003/ using document root <none> and password file <none>
I20260927 07:09:53.697956 14697 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:53.698004 14697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:53.698123 14697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:53.699196 14697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "390453c054bd46cbb2a5fa5df45f02c7"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "a156f7f26872458bc0110b404b12c9c0"
server_key_iv: "b75062adfb33e032248936265c954269"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.699513 14697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "390453c054bd46cbb2a5fa5df45f02c7"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "a156f7f26872458bc0110b404b12c9c0"
server_key_iv: "b75062adfb33e032248936265c954269"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.700717 14697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:09:53.701347 14711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.701509 14697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:53.701587 14697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "390453c054bd46cbb2a5fa5df45f02c7"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "a156f7f26872458bc0110b404b12c9c0"
server_key_iv: "b75062adfb33e032248936265c954269"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.701642 14697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:53.704939 14697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:53.705168 14697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:53.705328 14697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:53.705521 14697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:53.705793 14697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:53.705824 14697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.705848 14697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:53.705873 14697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.711045 14697 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:42205
I20260927 07:09:53.711097 14824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:42205 every 8 connection(s)
I20260927 07:09:53.711361 14697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:09:53.712128 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 14697
I20260927 07:09:53.712218 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:53.712427 4793 external_mini_cluster.cc:1428] Setting key 8b7cddd842586fa1ea3b216a6138e3ea
I20260927 07:09:53.713385 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33485
--builtin_ntp_servers=127.4.174.84:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 07:09:53.714298 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.712006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:60873 (local address 127.4.174.126:33485)
0927 07:09:53.712056 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.712058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.712298 (+ 240us) server_negotiation.cc:407] Connection header received
0927 07:09:53.712874 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.712875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.712880 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.712916 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.714110 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:53.714111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:53.714111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:53.714115 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:53.714129 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:53.714133 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:53.714176 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:53.714237 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 07:09:53.714247 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 07:09:53.714511 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.711997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33485 (local address 127.4.174.66:60873)
0927 07:09:53.712166 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:53.712210 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:53.712782 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:53.712987 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:53.712991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:53.713001 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:53.713957 (+ 956us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:53.713958 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:53.713996 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:53.714032 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:53.714177 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:53.714180 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:53.714183 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:53.714243 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:09:53.714303 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260927 07:09:53.715106 14825 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33485
I20260927 07:09:53.715189 14825 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:53.715323 14825 heartbeater.cc:511] Master 127.4.174.126:33485 requested a full tablet report, sending...
I20260927 07:09:53.715667 14506 ts_manager.cc:194] Registered new tserver with Master: 390453c054bd46cbb2a5fa5df45f02c7 (127.4.174.66:42205)
W20260927 07:09:53.792186 14828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:53.792387 14828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:53.792409 14828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:53.793567 14828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:53.793607 14828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:53.793632 14828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:53.795003 14828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:53.795222 14828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:53.795499 14828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:53.800254 14833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.800297 14836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:53.800292 14834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:53.800498 14828 server_base.cc:1034] running on GCE node
I20260927 07:09:53.800716 14828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:53.800937 14828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:53.802073 14828 hybrid_clock.cc:648] HybridClock initialized: now 1790492993802059 us; error 30 us; skew 500 ppm
I20260927 07:09:53.803344 14828 webserver.cc:460] Webserver started at http://127.4.174.67:43175/ using document root <none> and password file <none>
I20260927 07:09:53.803612 14828 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:53.803659 14828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:53.803786 14828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:53.804883 14828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "e7508fd4c2374dd0a957405e4b87c135"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "3b5e926eaf9893480f326cd098cda8cb"
server_key_iv: "800a6289a9daec3ac4127e5d28bbc978"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.805222 14828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "e7508fd4c2374dd0a957405e4b87c135"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "3b5e926eaf9893480f326cd098cda8cb"
server_key_iv: "800a6289a9daec3ac4127e5d28bbc978"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.806373 14828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:53.807092 14842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.807248 14828 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.807328 14828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "e7508fd4c2374dd0a957405e4b87c135"
format_stamp: "Formatted at 2026-09-27 07:09:53 on dist-test-slave-zw7p"
server_key: "3b5e926eaf9893480f326cd098cda8cb"
server_key_iv: "800a6289a9daec3ac4127e5d28bbc978"
server_key_version: "encryptionkey@0"
I20260927 07:09:53.807381 14828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:53.810653 14828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:53.810868 14828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:53.810961 14828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:53.811115 14828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:53.811357 14828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:53.811384 14828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.811403 14828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:53.811419 14828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:53.816532 14828 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:33521
I20260927 07:09:53.816596 14955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:33521 every 8 connection(s)
I20260927 07:09:53.816874 14828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:09:53.818042 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 14828
I20260927 07:09:53.818133 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:53.818318 4793 external_mini_cluster.cc:1428] Setting key 1174b84485b2b962251846fab2e782e1
I20260927 07:09:53.820005 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.817556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:40159 (local address 127.4.174.126:33485)
0927 07:09:53.817604 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.817606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.817865 (+ 259us) server_negotiation.cc:407] Connection header received
0927 07:09:53.818498 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.818500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.818506 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.818610 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.819813 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:53.819814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:53.819815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:09:53.819818 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:53.819833 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:09:53.819837 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:53.819886 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:53.819939 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 07:09:53.819951 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:09:53.820196 14958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:53.817552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:33485 (local address 127.4.174.67:40159)
0927 07:09:53.817746 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:53.817780 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:53.818389 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:53.818683 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:53.818687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:53.818697 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:53.819669 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:09:53.819670 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:53.819698 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:09:53.819728 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:53.819885 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:53.819887 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:53.819890 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:09:53.819940 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:09:53.819989 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260927 07:09:53.820609 14956 heartbeater.cc:347] Connected to a master server at 127.4.174.126:33485
I20260927 07:09:53.820693 14956 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:53.820819 14956 heartbeater.cc:511] Master 127.4.174.126:33485 requested a full tablet report, sending...
I20260927 07:09:53.821143 14506 ts_manager.cc:194] Registered new tserver with Master: e7508fd4c2374dd0a957405e4b87c135 (127.4.174.67:33521)
I20260927 07:09:53.830004 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 07:09:53.831755 14546 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:09:53.831286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.4.174.126:33485)
0927 07:09:53.831331 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:53.831333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:53.831425 (+ 92us) server_negotiation.cc:407] Connection header received
0927 07:09:53.831515 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:53.831516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:53.831521 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:53.831561 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:53.831682 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48254: BlockingRecv error: recv got EOF from 127.0.0.1:48254 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:09:53.831807 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 14566
I20260927 07:09:53.835758 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 14697
I20260927 07:09:53.839355 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 14828
I20260927 07:09:53.842730 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 14475
2026-09-27T07:09:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (542 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:53 dist-test-slave-zw7p krb5kdc[14966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:53 dist-test-slave-zw7p krb5kdc[14966](info): set up 2 sockets
Sep 27 07:09:53 dist-test-slave-zw7p krb5kdc[14966](info): commencing operation
krb5kdc: starting...
W20260927 07:09:54.787091 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:54 dist-test-slave-zw7p krb5kdc[14966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:09:54.899497 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:44109
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:44109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:54.979664 14979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:54.979880 14979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:54.979903 14979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:54.981036 14979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:09:54.981074 14979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:54.981087 14979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:54.981105 14979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:09:54.981128 14979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:09:54.982599 14979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:44109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:44109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:54.982820 14979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:54.983024 14979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:54.987444 14984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:54.987516 14979 server_base.cc:1034] running on GCE node
W20260927 07:09:54.987582 14985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:54.987556 14987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:54.987900 14979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:54.988170 14979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:54.989307 14979 hybrid_clock.cc:648] HybridClock initialized: now 1790492994989294 us; error 29 us; skew 500 ppm
Sep 27 07:09:54 dist-test-slave-zw7p krb5kdc[14966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492994, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:54.991596 14979 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:09:54.991984 14979 webserver.cc:460] Webserver started at http://127.4.174.126:40513/ using document root <none> and password file <none>
I20260927 07:09:54.992154 14979 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:54.992194 14979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:54.992278 14979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:54.993290 14979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "0c9acf09fd384590881f89b574c1ba7e"
format_stamp: "Formatted at 2026-09-27 07:09:54 on dist-test-slave-zw7p"
server_key: "68a903f52ac41c749a16d1bd57b950f9"
server_key_iv: "bab5bd4d824e1dd1c60fdd12531c0fbf"
server_key_version: "encryptionkey@0"
I20260927 07:09:54.993588 14979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "0c9acf09fd384590881f89b574c1ba7e"
format_stamp: "Formatted at 2026-09-27 07:09:54 on dist-test-slave-zw7p"
server_key: "68a903f52ac41c749a16d1bd57b950f9"
server_key_iv: "bab5bd4d824e1dd1c60fdd12531c0fbf"
server_key_version: "encryptionkey@0"
I20260927 07:09:54.994818 14979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:09:54.995465 14994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:54.995612 14979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:54.995694 14979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "0c9acf09fd384590881f89b574c1ba7e"
format_stamp: "Formatted at 2026-09-27 07:09:54 on dist-test-slave-zw7p"
server_key: "68a903f52ac41c749a16d1bd57b950f9"
server_key_iv: "bab5bd4d824e1dd1c60fdd12531c0fbf"
server_key_version: "encryptionkey@0"
I20260927 07:09:54.995752 14979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:55.003080 14979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:55.005079 14979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:55.005206 14979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:55.008847 14979 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:44109
I20260927 07:09:55.008888 15046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:44109 every 8 connection(s)
I20260927 07:09:55.009254 14979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:09:55.010130 15047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:55.012539 15047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e: Bootstrap starting.
I20260927 07:09:55.013129 15047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:55.013413 15047 log.cc:824] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:55.014072 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 14979
I20260927 07:09:55.014117 15047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e: No bootstrap required, opened a new log
I20260927 07:09:55.014160 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:09:55.014314 4793 external_mini_cluster.cc:1428] Setting key 428329df00ee365eb03cfb977d937ad3
I20260927 07:09:55.015949 15047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9acf09fd384590881f89b574c1ba7e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44109 } }
I20260927 07:09:55.016083 15047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:55.016115 15047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9acf09fd384590881f89b574c1ba7e, State: Initialized, Role: FOLLOWER
I20260927 07:09:55.016260 15047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9acf09fd384590881f89b574c1ba7e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44109 } }
I20260927 07:09:55.016351 15047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:09:55.016388 15047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:09:55.016427 15047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:55.017035 15047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9acf09fd384590881f89b574c1ba7e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44109 } }
I20260927 07:09:55.017124 15047 leader_election.cc:302] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c9acf09fd384590881f89b574c1ba7e; no voters:
I20260927 07:09:55.017272 15047 leader_election.cc:288] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:09:55.017306 15052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:55.017408 15052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [term 1 LEADER]: Becoming Leader. State: Replica: 0c9acf09fd384590881f89b574c1ba7e, State: Running, Role: LEADER
I20260927 07:09:55.017485 15052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9acf09fd384590881f89b574c1ba7e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44109 } }
I20260927 07:09:55.017531 15047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:09:55.017783 15052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c9acf09fd384590881f89b574c1ba7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9acf09fd384590881f89b574c1ba7e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44109 } } }
I20260927 07:09:55.017864 15052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:55.017982 15052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c9acf09fd384590881f89b574c1ba7e. Latest consensus state: current_term: 1 leader_uuid: "0c9acf09fd384590881f89b574c1ba7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9acf09fd384590881f89b574c1ba7e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 44109 } } }
I20260927 07:09:55.018039 15052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e [sys.catalog]: This master's current role is: LEADER
I20260927 07:09:55.018159 15056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:09:55.018692 15056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:09:55.020284 15056 catalog_manager.cc:1347] Generated new cluster ID: 8a5fc8222244495abdf532d0058287d9
I20260927 07:09:55.020339 15056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[14966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:55.026068 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.014840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.4.174.126:44109)
0927 07:09:55.014999 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.015009 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.015036 (+ 27us) server_negotiation.cc:407] Connection header received
0927 07:09:55.015926 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.015933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.015984 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.016103 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:55.016819 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.017989 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.019143 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.019652 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.019746 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.022267 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:55.022271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:55.022271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:55.022283 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:55.024248 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.024626 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.024629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.024630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.024739 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.024906 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.024907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.024907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.025184 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:55.025647 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.025713 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.025853 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260927 07:09:55.056185 15056 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:09:55.056717 15056 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:09:55.067368 15056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e: Generated new TSK 0
I20260927 07:09:55.067574 15056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:09:55.097585 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44109
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:55.197862 15073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:55.198074 15073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:55.198097 15073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:55.199445 15073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:55.199491 15073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:55.199520 15073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:09:55.200992 15073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:55.201222 15073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:55.201442 15073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 07:09:55.206526 15073 server_base.cc:1034] running on GCE node
W20260927 07:09:55.206488 15079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:55.206501 15078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:55.206491 15081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:55.206910 15073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:55.207129 15073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:55.208284 15073 hybrid_clock.cc:648] HybridClock initialized: now 1790492995208267 us; error 28 us; skew 500 ppm
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[14966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492995, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:55.210722 15073 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:09:55.211125 15073 webserver.cc:460] Webserver started at http://127.4.174.65:34181/ using document root <none> and password file <none>
I20260927 07:09:55.211315 15073 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:55.211355 15073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:55.211449 15073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:55.212424 15073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "06b5ad05d4a9496abb20d9e62ad66e0a"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "b6690dcb724556865d7ca4af6afa1efd"
server_key_iv: "1966759002eb2191a3d61bea46ed03c8"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.212715 15073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "06b5ad05d4a9496abb20d9e62ad66e0a"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "b6690dcb724556865d7ca4af6afa1efd"
server_key_iv: "1966759002eb2191a3d61bea46ed03c8"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.213917 15073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:55.214633 15088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.214789 15073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:09:55.214870 15073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "06b5ad05d4a9496abb20d9e62ad66e0a"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "b6690dcb724556865d7ca4af6afa1efd"
server_key_iv: "1966759002eb2191a3d61bea46ed03c8"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.214931 15073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:55.226065 15073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:55.227981 15073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:55.228132 15073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:55.228365 15073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:55.228677 15073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:55.228719 15073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.228753 15073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:55.228787 15073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.234815 15073 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:34095
I20260927 07:09:55.234874 15201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:34095 every 8 connection(s)
I20260927 07:09:55.235183 15073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[14966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492995, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:55.243284 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 15073
I20260927 07:09:55.243363 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:09:55.243546 4793 external_mini_cluster.cc:1428] Setting key 9c4327e1586f7cac77568e8540d034d7
I20260927 07:09:55.244330 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.235872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:45421 (local address 127.4.174.126:44109)
0927 07:09:55.235965 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.235967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.236139 (+ 172us) server_negotiation.cc:407] Connection header received
0927 07:09:55.236656 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.236657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.236668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.236721 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:55.237609 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.238117 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.239089 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.239300 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.239354 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.241567 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:55.241568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:55.241569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:55.241577 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:55.242856 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.243272 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.243273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.243273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.243337 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.243669 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.243670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.243670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.243813 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:55.243871 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.244147 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.244245 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 07:09:55.244522 15204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.235845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44109 (local address 127.4.174.65:45421)
0927 07:09:55.236003 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.236039 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:55.236541 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:55.236780 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:55.236783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:55.237328 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:55.237526 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.237531 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.238182 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.238183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.238993 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.238996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.239395 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.239396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.239515 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.240083 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:55.240091 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:55.241478 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:55.242933 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:55.242940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:55.242942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:55.243193 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:55.243453 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:55.243454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:55.243454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:55.243598 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:55.243873 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:55.243879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:55.244026 (+ 147us) client_negotiation.cc:769] Sending connection context
0927 07:09:55.244116 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 07:09:55.244284 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260927 07:09:55.244943 15202 heartbeater.cc:347] Connected to a master server at 127.4.174.126:44109
I20260927 07:09:55.245036 15202 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:55.245200 15202 heartbeater.cc:511] Master 127.4.174.126:44109 requested a full tablet report, sending...
I20260927 07:09:55.245684 15011 ts_manager.cc:194] Registered new tserver with Master: 06b5ad05d4a9496abb20d9e62ad66e0a (127.4.174.65:34095)
I20260927 07:09:55.246263 15011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:45421
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:09:55.265759 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44109
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:55.344424 15207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:55.344641 15207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:55.344663 15207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:55.345767 15207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:55.345806 15207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:55.345830 15207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:09:55.347213 15207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:55.347391 15207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:55.347560 15207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:55.352296 15212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:55.352413 15215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:55.352376 15213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:55.352427 15207 server_base.cc:1034] running on GCE node
I20260927 07:09:55.352730 15207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:55.352938 15207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:55.354092 15207 hybrid_clock.cc:648] HybridClock initialized: now 1790492995354075 us; error 32 us; skew 500 ppm
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[14966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492995, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:55.356532 15207 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:09:55.356962 15207 webserver.cc:460] Webserver started at http://127.4.174.66:38017/ using document root <none> and password file <none>
I20260927 07:09:55.357156 15207 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:55.357203 15207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:55.357328 15207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:55.358335 15207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "040ae3dc033446269ac04988959de316"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "5670a64be53934a0638841f7e41653f8"
server_key_iv: "e46c7519e4fa9243869985f8bceb3a1f"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.358688 15207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "040ae3dc033446269ac04988959de316"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "5670a64be53934a0638841f7e41653f8"
server_key_iv: "e46c7519e4fa9243869985f8bceb3a1f"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.359959 15207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:09:55.360632 15222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.360778 15207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:09:55.360865 15207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "040ae3dc033446269ac04988959de316"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "5670a64be53934a0638841f7e41653f8"
server_key_iv: "e46c7519e4fa9243869985f8bceb3a1f"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.360931 15207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:55.371891 15207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:55.374025 15207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:55.374161 15207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:55.374388 15207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:55.374717 15207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:55.374752 15207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.374782 15207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:55.374815 15207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.380159 15207 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:43235
I20260927 07:09:55.380225 15335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:43235 every 8 connection(s)
I20260927 07:09:55.380513 15207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[14966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492995, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:55.389608 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.381234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:57503 (local address 127.4.174.126:44109)
0927 07:09:55.381297 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.381299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.381471 (+ 172us) server_negotiation.cc:407] Connection header received
0927 07:09:55.381966 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.381967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.381978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.382055 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:55.383001 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.383502 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.384471 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.384664 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.384718 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.386941 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:55.386943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:55.386943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:55.386951 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:55.388097 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.388501 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.388503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.388503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.388575 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.388869 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.388870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.388870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.389006 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:55.389082 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.389459 (+ 377us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.389537 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 07:09:55.389766 15338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.381202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44109 (local address 127.4.174.66:57503)
0927 07:09:55.381348 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.381390 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:55.381877 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:55.382113 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:55.382117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:55.382700 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:55.382914 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.382919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.383580 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.383581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.384395 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.384397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.384759 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.384760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.384890 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.385430 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:55.385439 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:55.386853 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:55.388180 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:55.388187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:55.388189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:55.388410 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:55.388662 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:55.388663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:55.388664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:55.388773 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:55.389156 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:55.389163 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:55.389321 (+ 158us) client_negotiation.cc:769] Sending connection context
0927 07:09:55.389419 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 07:09:55.389555 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":85,"threads_started":1}
I20260927 07:09:55.390100 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 15207
I20260927 07:09:55.390201 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:09:55.390312 15336 heartbeater.cc:347] Connected to a master server at 127.4.174.126:44109
I20260927 07:09:55.390424 4793 external_mini_cluster.cc:1428] Setting key 7c5a8c61cf131e8a49a26bddce3c79d2
I20260927 07:09:55.390427 15336 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:55.390655 15336 heartbeater.cc:511] Master 127.4.174.126:44109 requested a full tablet report, sending...
I20260927 07:09:55.391078 15010 ts_manager.cc:194] Registered new tserver with Master: 040ae3dc033446269ac04988959de316 (127.4.174.66:43235)
I20260927 07:09:55.391546 15010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:57503
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:09:55.413980 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44109
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:09:55.494980 15341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:09:55.495210 15341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:09:55.495247 15341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:09:55.496436 15341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:09:55.496484 15341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:09:55.496522 15341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:09:55.498051 15341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:44109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:09:55.498347 15341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:09:55.498605 15341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:09:55.503506 15349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:55.503594 15341 server_base.cc:1034] running on GCE node
W20260927 07:09:55.503494 15347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:09:55.503494 15346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:09:55.503870 15341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:09:55.504087 15341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:09:55.505234 15341 hybrid_clock.cc:648] HybridClock initialized: now 1790492995505218 us; error 38 us; skew 500 ppm
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[14966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492995, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:09:55.507902 15341 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:09:55.508332 15341 webserver.cc:460] Webserver started at http://127.4.174.67:36961/ using document root <none> and password file <none>
I20260927 07:09:55.508538 15341 fs_manager.cc:359] Metadata directory not provided
I20260927 07:09:55.508584 15341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:09:55.508703 15341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:09:55.509697 15341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "f5be49e8c9ba470eaf2b4222a509f97c"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "cfbad36c49207d943e5605305e076fce"
server_key_iv: "fd446408e870de2b5b140e8cddb0f9f1"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.510017 15341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "f5be49e8c9ba470eaf2b4222a509f97c"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "cfbad36c49207d943e5605305e076fce"
server_key_iv: "fd446408e870de2b5b140e8cddb0f9f1"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.511296 15341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:09:55.511988 15356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.512147 15341 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:09:55.512231 15341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "f5be49e8c9ba470eaf2b4222a509f97c"
format_stamp: "Formatted at 2026-09-27 07:09:55 on dist-test-slave-zw7p"
server_key: "cfbad36c49207d943e5605305e076fce"
server_key_iv: "fd446408e870de2b5b140e8cddb0f9f1"
server_key_version: "encryptionkey@0"
I20260927 07:09:55.512331 15341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:09:55.521955 15341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:09:55.523865 15341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:09:55.524003 15341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:09:55.524219 15341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:09:55.524513 15341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:09:55.524544 15341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.524573 15341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:09:55.524596 15341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:09:55.530063 15341 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:35925
I20260927 07:09:55.530112 15469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:35925 every 8 connection(s)
I20260927 07:09:55.530426 15341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[14966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492995, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:09:55.538681 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 15341
I20260927 07:09:55.538771 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:09:55.538964 4793 external_mini_cluster.cc:1428] Setting key e590f946630a57be147c2f1a742d45e4
I20260927 07:09:55.539055 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.531096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:36457 (local address 127.4.174.126:44109)
0927 07:09:55.531144 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.531146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.531347 (+ 201us) server_negotiation.cc:407] Connection header received
0927 07:09:55.531846 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.531847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.531858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.531901 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:55.532785 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.533275 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.534212 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.534401 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.534444 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.536521 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:55.536523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:55.536523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:55.536530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:55.537731 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.538082 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.538083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.538083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.538129 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.538310 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.538310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.538311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.538423 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:55.538470 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.538890 (+ 420us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.538961 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:09:55.539294 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.531066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:44109 (local address 127.4.174.67:36457)
0927 07:09:55.531218 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.531260 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:55.531746 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:55.531967 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:55.531971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:55.532525 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:09:55.532717 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.532723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.533339 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.533340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.534153 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.534155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.534462 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.534463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.534590 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.535043 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:09:55.535052 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:09:55.536430 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:09:55.537812 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:55.537818 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:55.537820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:55.538026 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:55.538173 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:09:55.538174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:09:55.538174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:09:55.538267 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:09:55.538489 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:09:55.538493 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:09:55.538687 (+ 194us) client_negotiation.cc:769] Sending connection context
0927 07:09:55.538858 (+ 171us) client_negotiation.cc:240] Negotiation successful
0927 07:09:55.539052 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260927 07:09:55.539731 15470 heartbeater.cc:347] Connected to a master server at 127.4.174.126:44109
I20260927 07:09:55.539830 15470 heartbeater.cc:464] Registering TS with master...
I20260927 07:09:55.540011 15470 heartbeater.cc:511] Master 127.4.174.126:44109 requested a full tablet report, sending...
I20260927 07:09:55.540382 15010 ts_manager.cc:194] Registered new tserver with Master: f5be49e8c9ba470eaf2b4222a509f97c (127.4.174.67:35925)
I20260927 07:09:55.540793 15010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:36457
I20260927 07:09:55.550869 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:09:55.557490 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.552038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.4.174.126:44109)
0927 07:09:55.552097 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.552099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.552185 (+ 86us) server_negotiation.cc:407] Connection header received
0927 07:09:55.552250 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.552251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.552261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.552314 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:09:55.552831 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.553318 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.554031 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.554231 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.554283 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.555416 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:09:55.555418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:09:55.555418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:09:55.555426 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:09:55.556487 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.556773 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.556775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.556775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.556829 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:09:55.556984 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:09:55.556985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:09:55.556986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:09:55.557131 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:09:55.557216 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.557328 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.557399 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 07:09:55.558920 15010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:09:55.560429 15010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:09:55.566437 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.562312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:35925 (local address 127.0.0.1:57744)
0927 07:09:55.562633 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.562674 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:55.562737 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:55.562978 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:55.562981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:55.562983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:55.563034 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.563036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.564161 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.564162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.565354 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.565357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.566180 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.566181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.566247 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.566256 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 07:09:55.566309 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 07:09:55.566362 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":104,"threads_started":1}
I20260927 07:09:55.566459 15480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.562408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.4.174.67:35925)
0927 07:09:55.562830 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.562834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.562843 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:55.562894 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.562896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.562908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.562966 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:55.563102 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.564080 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.565449 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.566060 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.566095 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.566136 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.566349 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.566391 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":81,"threads_started":1}
I20260927 07:09:55.566929 15478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.562321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:34095 (local address 127.0.0.1:54452)
0927 07:09:55.562635 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.562674 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:55.562738 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:55.563496 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:55.563496 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:55.563498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:55.563535 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.563536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.564760 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.564761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.565617 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.565620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.566697 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.566699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.566759 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.566764 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:55.566815 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 07:09:55.566864 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":58,"threads_started":1}
I20260927 07:09:55.566959 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.562436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54452 (local address 127.4.174.65:34095)
0927 07:09:55.563351 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.563355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.563364 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:55.563398 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.563401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.563414 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.563491 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:55.563601 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.564689 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.565694 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.566437 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.566680 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.566748 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.566826 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.566868 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":78,"threads_started":1}
I20260927 07:09:55.567250 15404 tablet_service.cc:1467] Processing CreateTablet for tablet b14180f21f714a69845b7e4d1c2bdf31 (DEFAULT_TABLE table=test-table [id=fc781853cfe24606a4b34e88eea24105]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:55.567448 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.562647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45280 (local address 127.4.174.66:43235)
0927 07:09:55.563093 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.563098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.563107 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:09:55.563141 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.563143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.563157 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.563226 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:55.563394 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.564505 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.565735 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.566439 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.566504 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.566788 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.567315 (+ 527us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.567363 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":69,"threads_started":1}
I20260927 07:09:55.567590 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.562527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43235 (local address 127.0.0.1:45280)
0927 07:09:55.562633 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.562674 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:55.562737 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:55.563271 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:55.563272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:55.563275 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:55.563316 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.563318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.564603 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.564604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.565672 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.565674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.566824 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.566825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.566886 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.566891 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:09:55.566940 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 07:09:55.566980 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":32,"thread_start_us":59,"threads_started":1}
I20260927 07:09:55.567623 15404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14180f21f714a69845b7e4d1c2bdf31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:55.567731 15136 tablet_service.cc:1467] Processing CreateTablet for tablet b14180f21f714a69845b7e4d1c2bdf31 (DEFAULT_TABLE table=test-table [id=fc781853cfe24606a4b34e88eea24105]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:55.568037 15136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14180f21f714a69845b7e4d1c2bdf31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:55.568156 15270 tablet_service.cc:1467] Processing CreateTablet for tablet b14180f21f714a69845b7e4d1c2bdf31 (DEFAULT_TABLE table=test-table [id=fc781853cfe24606a4b34e88eea24105]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:09:55.568403 15270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14180f21f714a69845b7e4d1c2bdf31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:09:55.570787 15484 tablet_bootstrap.cc:492] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Bootstrap starting.
I20260927 07:09:55.571081 15485 tablet_bootstrap.cc:492] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Bootstrap starting.
I20260927 07:09:55.571445 15484 tablet_bootstrap.cc:654] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:55.571655 15485 tablet_bootstrap.cc:654] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:55.572036 15484 log.cc:824] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:55.572199 15485 log.cc:824] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:55.572494 15486 tablet_bootstrap.cc:492] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a: Bootstrap starting.
I20260927 07:09:55.572813 15484 tablet_bootstrap.cc:492] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: No bootstrap required, opened a new log
I20260927 07:09:55.572845 15485 tablet_bootstrap.cc:492] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: No bootstrap required, opened a new log
I20260927 07:09:55.572893 15484 ts_tablet_manager.cc:1397] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:55.572901 15485 ts_tablet_manager.cc:1397] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:55.573096 15486 tablet_bootstrap.cc:654] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a: Neither blocks nor log segments found. Creating new log.
I20260927 07:09:55.573613 15486 log.cc:824] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a: Log is configured to *not* fsync() on all Append() calls
I20260927 07:09:55.574311 15486 tablet_bootstrap.cc:492] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a: No bootstrap required, opened a new log
I20260927 07:09:55.574390 15486 ts_tablet_manager.cc:1397] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:09:55.574548 15485 raft_consensus.cc:348] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.574730 15485 raft_consensus.cc:374] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:55.574769 15485 raft_consensus.cc:729] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5be49e8c9ba470eaf2b4222a509f97c, State: Initialized, Role: FOLLOWER
I20260927 07:09:55.574895 15485 consensus_queue.cc:260] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.574957 15484 raft_consensus.cc:348] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.575105 15484 raft_consensus.cc:374] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:55.575136 15484 raft_consensus.cc:729] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040ae3dc033446269ac04988959de316, State: Initialized, Role: FOLLOWER
I20260927 07:09:55.575137 15485 ts_tablet_manager.cc:1428] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:09:55.575232 15470 heartbeater.cc:503] Master 127.4.174.126:44109 was elected leader, sending a full tablet report...
I20260927 07:09:55.575258 15484 consensus_queue.cc:260] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.575495 15484 ts_tablet_manager.cc:1428] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 07:09:55.575605 15336 heartbeater.cc:503] Master 127.4.174.126:44109 was elected leader, sending a full tablet report...
I20260927 07:09:55.576267 15486 raft_consensus.cc:348] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.576427 15486 raft_consensus.cc:374] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:09:55.576457 15486 raft_consensus.cc:729] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b5ad05d4a9496abb20d9e62ad66e0a, State: Initialized, Role: FOLLOWER
I20260927 07:09:55.576547 15486 consensus_queue.cc:260] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.576792 15486 ts_tablet_manager.cc:1428] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 07:09:55.576844 15202 heartbeater.cc:503] Master 127.4.174.126:44109 was elected leader, sending a full tablet report...
W20260927 07:09:55.631155 15337 tablet.cc:2369] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:09:55.735934 15203 tablet.cc:2369] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:55.772428 15492 raft_consensus.cc:482] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:09:55.772557 15492 raft_consensus.cc:504] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.772904 15492 leader_election.cc:288] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5be49e8c9ba470eaf2b4222a509f97c (127.4.174.67:35925), 040ae3dc033446269ac04988959de316 (127.4.174.66:43235)
I20260927 07:09:55.776851 15480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.773170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:39925 (local address 127.4.174.67:35925)
0927 07:09:55.773219 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.773221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.773300 (+ 79us) server_negotiation.cc:407] Connection header received
0927 07:09:55.773398 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.773399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.773409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.773491 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:55.773650 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.774396 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.775887 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.776478 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.776554 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.776563 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.776735 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.776773 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:09:55.776851 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.773365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:58607 (local address 127.4.174.66:43235)
0927 07:09:55.773407 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.773409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.773446 (+ 37us) server_negotiation.cc:407] Connection header received
0927 07:09:55.773504 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.773505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.773512 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.773556 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:09:55.773703 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.774391 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.775887 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.776475 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.776554 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.776563 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.776735 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.776773 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:09:55.776895 15494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.773289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:43235 (local address 127.4.174.65:58607)
0927 07:09:55.773400 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.773408 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:55.773451 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:55.773571 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:55.773571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:55.773573 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:55.773645 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.773646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.774468 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.774469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.775788 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.775791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.776559 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.776560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.776664 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.776672 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:09:55.776725 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 07:09:55.776779 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":10,"thread_start_us":53,"threads_started":1}
I20260927 07:09:55.776897 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.773095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:35925 (local address 127.4.174.65:39925)
0927 07:09:55.773227 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.773238 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:09:55.773303 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:09:55.773504 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:09:55.773505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:09:55.773507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:09:55.773589 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.773591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.774474 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.774475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.775788 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:09:55.775791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.776558 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.776558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:09:55.776664 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.776670 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 07:09:55.776725 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 07:09:55.776779 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":60,"thread_start_us":76,"threads_started":1}
I20260927 07:09:55.777261 15290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14180f21f714a69845b7e4d1c2bdf31" candidate_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040ae3dc033446269ac04988959de316" is_pre_election: true
I20260927 07:09:55.777312 15424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14180f21f714a69845b7e4d1c2bdf31" candidate_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" is_pre_election: true
I20260927 07:09:55.777437 15424 raft_consensus.cc:2454] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b5ad05d4a9496abb20d9e62ad66e0a in term 0.
I20260927 07:09:55.777437 15290 raft_consensus.cc:2454] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b5ad05d4a9496abb20d9e62ad66e0a in term 0.
I20260927 07:09:55.777661 15091 leader_election.cc:302] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 040ae3dc033446269ac04988959de316, 06b5ad05d4a9496abb20d9e62ad66e0a; no voters:
I20260927 07:09:55.777787 15492 raft_consensus.cc:2781] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:09:55.777842 15492 raft_consensus.cc:482] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:09:55.777877 15492 raft_consensus.cc:3037] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:55.778539 15492 raft_consensus.cc:504] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.778694 15492 leader_election.cc:288] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [CANDIDATE]: Term 1 election: Requested vote from peers f5be49e8c9ba470eaf2b4222a509f97c (127.4.174.67:35925), 040ae3dc033446269ac04988959de316 (127.4.174.66:43235)
I20260927 07:09:55.778858 15424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14180f21f714a69845b7e4d1c2bdf31" candidate_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5be49e8c9ba470eaf2b4222a509f97c"
I20260927 07:09:55.778872 15290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14180f21f714a69845b7e4d1c2bdf31" candidate_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040ae3dc033446269ac04988959de316"
I20260927 07:09:55.778930 15424 raft_consensus.cc:3037] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:55.778954 15290 raft_consensus.cc:3037] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:09:55.779606 15424 raft_consensus.cc:2454] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b5ad05d4a9496abb20d9e62ad66e0a in term 1.
I20260927 07:09:55.779747 15290 raft_consensus.cc:2454] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b5ad05d4a9496abb20d9e62ad66e0a in term 1.
I20260927 07:09:55.779762 15091 leader_election.cc:302] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b5ad05d4a9496abb20d9e62ad66e0a, f5be49e8c9ba470eaf2b4222a509f97c; no voters:
I20260927 07:09:55.779838 15492 raft_consensus.cc:2781] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:09:55.779975 15492 raft_consensus.cc:686] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 1 LEADER]: Becoming Leader. State: Replica: 06b5ad05d4a9496abb20d9e62ad66e0a, State: Running, Role: LEADER
I20260927 07:09:55.780073 15492 consensus_queue.cc:237] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } }
I20260927 07:09:55.780839 15011 catalog_manager.cc:5495] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 06b5ad05d4a9496abb20d9e62ad66e0a (127.4.174.65). New cstate: current_term: 1 leader_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b5ad05d4a9496abb20d9e62ad66e0a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 34095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 } health_report { overall_health: UNKNOWN } } }
W20260927 07:09:55.780947 15471 tablet.cc:2369] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:09:55.790330 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.788309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54466 (local address 127.4.174.65:34095)
0927 07:09:55.788362 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 07:09:55.788363 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:09:55.788388 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:09:55.788414 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:09:55.788415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:09:55.788429 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:09:55.788484 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:09:55.788588 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.789032 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.789771 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:09:55.789958 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:09:55.790012 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:09:55.790104 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:09:55.790198 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:09:55.790250 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:09:55.790262 (+ 12us) server_negotiation.cc:299] Negotiation successful
0927 07:09:55.790276 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 07:09:55.791987 15424 raft_consensus.cc:1261] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 1 FOLLOWER]: Refusing update from remote peer 06b5ad05d4a9496abb20d9e62ad66e0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:55.791988 15290 raft_consensus.cc:1261] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 1 FOLLOWER]: Refusing update from remote peer 06b5ad05d4a9496abb20d9e62ad66e0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:09:55.792264 15492 consensus_queue.cc:1035] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "040ae3dc033446269ac04988959de316" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 43235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:55.792400 15495 consensus_queue.cc:1035] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5be49e8c9ba470eaf2b4222a509f97c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:09:55.794672 15499 mvcc.cc:205] Tried to move back new op lower bound from 7333859310761553920 to 7333859310715797504. Current Snapshot: MvccSnapshot[applied={T|T < 7333859310761553920}]
I20260927 07:09:55.795349 15500 mvcc.cc:205] Tried to move back new op lower bound from 7333859310761553920 to 7333859310715797504. Current Snapshot: MvccSnapshot[applied={T|T < 7333859310761553920}]
I20260927 07:09:55.796711 15011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:55.796804 15011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:09:55.797492 15011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c9acf09fd384590881f89b574c1ba7e: Sending DeleteTablet for 3 replicas of tablet b14180f21f714a69845b7e4d1c2bdf31
I20260927 07:09:55.797811 15136 tablet_service.cc:1514] Processing DeleteTablet for tablet b14180f21f714a69845b7e4d1c2bdf31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:55 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:54452
I20260927 07:09:55.797811 15270 tablet_service.cc:1514] Processing DeleteTablet for tablet b14180f21f714a69845b7e4d1c2bdf31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:55 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:45280
I20260927 07:09:55.798040 15507 tablet_replica.cc:330] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a: stopping tablet replica
I20260927 07:09:55.798139 15507 raft_consensus.cc:2229] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:09:55.798231 15507 raft_consensus.cc:2258] T b14180f21f714a69845b7e4d1c2bdf31 P 06b5ad05d4a9496abb20d9e62ad66e0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:55.798271 15404 tablet_service.cc:1514] Processing DeleteTablet for tablet b14180f21f714a69845b7e4d1c2bdf31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:09:55 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:57744
I20260927 07:09:55.798362 15508 tablet_replica.cc:330] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: stopping tablet replica
I20260927 07:09:55.798454 15508 raft_consensus.cc:2229] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:55.798478 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 15073
I20260927 07:09:55.798542 15508 raft_consensus.cc:2258] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:55.799217 15508 ts_tablet_manager.cc:1905] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:09:55.800776 15508 ts_tablet_manager.cc:1918] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:55.800825 15508 log.cc:1196] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/b14180f21f714a69845b7e4d1c2bdf31
I20260927 07:09:55.801000 15508 ts_tablet_manager.cc:1939] T b14180f21f714a69845b7e4d1c2bdf31 P 040ae3dc033446269ac04988959de316: Deleting consensus metadata
I20260927 07:09:55.801307 14997 catalog_manager.cc:4841] TS 040ae3dc033446269ac04988959de316 (127.4.174.66:43235): tablet b14180f21f714a69845b7e4d1c2bdf31 (table test-table [id=fc781853cfe24606a4b34e88eea24105]) successfully deleted
I20260927 07:09:55.802661 15509 tablet_replica.cc:330] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: stopping tablet replica
I20260927 07:09:55.802763 15509 raft_consensus.cc:2229] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:09:55.802865 15509 raft_consensus.cc:2258] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:09:55.803220 15509 ts_tablet_manager.cc:1905] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 07:09:55.804004 14995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:34095 (error 108)
I20260927 07:09:55.804203 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 15207
I20260927 07:09:55.805349 15509 ts_tablet_manager.cc:1918] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:09:55.805449 15509 log.cc:1196] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/b14180f21f714a69845b7e4d1c2bdf31
I20260927 07:09:55.805685 15509 ts_tablet_manager.cc:1939] T b14180f21f714a69845b7e4d1c2bdf31 P f5be49e8c9ba470eaf2b4222a509f97c: Deleting consensus metadata
I20260927 07:09:55.806071 14997 catalog_manager.cc:4841] TS f5be49e8c9ba470eaf2b4222a509f97c (127.4.174.67:35925): tablet b14180f21f714a69845b7e4d1c2bdf31 (table test-table [id=fc781853cfe24606a4b34e88eea24105]) successfully deleted
I20260927 07:09:55.806931 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:09:55.806791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:34095 (local address 127.0.0.1:54476)
0927 07:09:55.806845 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 07:09:55.806896 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:34095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 07:09:55.807061 14995 catalog_manager.cc:4584] TS 06b5ad05d4a9496abb20d9e62ad66e0a (127.4.174.65:34095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b14180f21f714a69845b7e4d1c2bdf31: Network error: Client connection negotiation failed: client connection to 127.4.174.65:34095: connect: Connection refused (error 111)
I20260927 07:09:55.809437 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 15341
I20260927 07:09:55.812992 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 14979
2026-09-27T07:09:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1970 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[15513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[15513](info): set up 2 sockets
Sep 27 07:09:55 dist-test-slave-zw7p krb5kdc[15513](info): commencing operation
krb5kdc: starting...
W20260927 07:09:56.767228 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:09:56 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:09:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:09:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.174.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.174.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/rangeradmin_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/rangeradmin_127.4.174.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.174.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.174.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/rangerlookup_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/rangerlookup_127.4.174.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.174.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.174.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/HTTP_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/HTTP_127.4.174.84@KRBTEST.COM.keytab.
I20260927 07:09:56.920900 4793 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIteyrz/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/postgres -l logfile start
2026-09-27 07:09:59.260 UTC [15543] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 07:09:59.261 UTC [15543] LOG: listening on IPv4 address "127.4.174.84", port 48875
2026-09-27 07:09:59.264 UTC [15543] LOG: listening on Unix socket "/tmp/.s.PGSQL.48875"
2026-09-27 07:09:59.290 UTC [15546] LOG: database system was shut down at 2026-09-27 07:09:57 UTC
2026-09-27 07:09:59.293 UTC [15543] LOG: database system is ready to accept connections
I20260927 07:10:00.155121 4793 mini_postgres.cc:96] Postgres bound to 48875
2026-09-27 07:10:00.161 UTC [15554] FATAL: database "slave" does not exist
127.4.174.84:48875 - accepting connections
I20260927 07:10:00.161782 4793 mini_ranger.cc:161] Starting Ranger...
I20260927 07:10:00.175665 4793 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 07:10:00.634790 4793 mini_ranger.cc:87] Created ranger Postgres database
I20260927 07:10:00.634871 4793 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-admin
2026-09-27 07:10:00,888 [I] DB FLAVOR :POSTGRES
2026-09-27 07:10:00,889 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 07:10:00,889 [I] Checking connection..
2026-09-27 07:10:00,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 07:10:01,142 [I] Checking connection passed.
2026-09-27 07:10:01,142 [I] --------- Verifying version history table ---------
2026-09-27 07:10:01,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:10:01,378 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 07:10:01,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:10:01,634 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 07:10:01,634 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 07:10:01,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 07:10:01.877 UTC [15618] WARNING: there is no transaction in progress
2026-09-27 07:10:01,882 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 07:10:01,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:10:02,141 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 07:10:02,141 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 07:10:02,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 07:10:02,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 07:10:02,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zw7p.c.gcp-upstream.internal','N') ;"
2026-09-27 07:10:02,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 07:10:03,078 [I] Table x_portal_user does not exist in database ranger
2026-09-27 07:10:03,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 07:10:03,320 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 07:10:03,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 07:10:03,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 07:10:03,814 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 07:10:03,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 07:10:04.729 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.744 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.760 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.776 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.793 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.811 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.865 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.872 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.887 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.898 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.908 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.924 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.934 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.941 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.947 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:04.954 UTC [15726] WARNING: there is no transaction in progress
2026-09-27 07:10:05,589 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 07:10:05,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 07:10:05,815 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 07:10:05,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 07:10:06,064 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 07:10:06,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 07:10:06,316 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 07:10:06,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 07:10:06,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 07:10:06,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zw7p.c.gcp-upstream.internal';"
2026-09-27 07:10:07,050 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 07:10:07,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 07:10:07,275 [I] DB_PATCHES have already been applied
I20260927 07:10:07.280964 4793 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIteyrz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 07:10:07.281033 4793 mini_ranger.cc:193] Using host: 127.4.174.84
I20260927 07:10:07.283949 4793 mini_ranger.cc:227] Ranger admin URL: http://127.4.174.84:38279
Sep 27, 2026 7:10:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 7:10:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 7:10:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 7:10:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 7:10:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 7:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 7:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 07:10:08.278204 4793 mini_ranger.cc:160] Time spent starting Ranger: real 8.116s user 0.000s sys 0.009s
Sep 27, 2026 7:10:28 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 7:10:28 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 7:10:28 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 7:10:28 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 7:10:29 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 07:10:29.709890 4793 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.174.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.174.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/rangerkms_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/rangerkms_127.4.174.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.174.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.174.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.174.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/HTTP_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.174.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/HTTP_127.4.174.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 07:10:29 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493029, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 07:10:29.792857 4793 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 07:10:29.802234 4793 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 07:10:30.615489 4793 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 07:10:30.615568 4793 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms
I20260927 07:10:30.615583 4793 mini_ranger_kms.cc:227] Using postgres at 127.4.174.84:48875
2026-09-27 07:10:30,775 [I] DB FLAVOR :POSTGRES
2026-09-27 07:10:30,776 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 07:10:30,776 [I] Checking connection
2026-09-27 07:10:30,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 07:10:31,015 [I] connection success
2026-09-27 07:10:31,015 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 07:10:31,015 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 07:10:31,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 07:10:31,247 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 07:10:31,247 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 07:10:31,247 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 07:10:31,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:48875/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 07:10:31,492 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 07:10:31.926169 4793 mini_ranger_kms.cc:310] Created kms service
I20260927 07:10:32.139652 4793 mini_ranger_kms.cc:326] Created kudu user
I20260927 07:10:32.179384 4793 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 07:10:32 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493029, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:10:32.559404 4793 mini_ranger_kms.cc:384] Added ranger policy
I20260927 07:10:32.559521 4793 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 07:10:32.559541 4793 mini_ranger_kms.cc:240] Using host: 127.4.174.84
I20260927 07:10:32.562961 4793 mini_ranger_kms.cc:276] Ranger KMS PID: 15906
I20260927 07:10:32.563097 4793 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.174.84:42301
Sep 27, 2026 7:10:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 7:10:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 7:10:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 7:10:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 7:10:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 7:10:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 07:10:34.634166 4793 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.841s user 0.006s sys 0.137s
I20260927 07:10:34.634263 4793 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 07:10:34.634336 4793 mini_ranger_kms.cc:401] 127.4.174.84:42301/kms/v1/keys
Sep 27 07:10:34 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790493034, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.174.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 07:10:35 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493034, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.174.84@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
Sep 27 07:10:35 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493034, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.174.84@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
Sep 27 07:10:37 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:10:37.911023 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:42649
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:42649
--ranger_config_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42301/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:10:37.997918 15957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:10:37.998124 15957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:10:37.998145 15957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:10:37.999353 15957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:10:37.999392 15957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:10:37.999405 15957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:10:37.999416 15957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:10:37.999426 15957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:10:38.001012 15957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.174.84:42301/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:42649
--ranger_config_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:42649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:10:38.001243 15957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:10:38.001464 15957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:10:38.006006 15962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:10:38.006016 15963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:10:38.006019 15965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:10:38.006299 15957 server_base.cc:1034] running on GCE node
I20260927 07:10:38.006524 15957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:10:38.006822 15957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:10:38.007969 15957 hybrid_clock.cc:648] HybridClock initialized: now 1790493038007957 us; error 29 us; skew 500 ppm
Sep 27 07:10:38 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493038, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:10:38.010340 15957 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:10:38.010798 15957 webserver.cc:460] Webserver started at http://127.4.174.126:35999/ using document root <none> and password file <none>
I20260927 07:10:38.011015 15957 fs_manager.cc:359] Metadata directory not provided
I20260927 07:10:38.011056 15957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:10:38.011178 15957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:10:38 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493038, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:10:38.068845 15957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "55cd82ae10d04176bbcfa5347df04db7"
format_stamp: "Formatted at 2026-09-27 07:10:38 on dist-test-slave-zw7p"
server_key: "b51fcaaf4e40b57bf4258eea0ef1f17d"
server_key_iv: "ec92f07eff2f1e4fd5569b7a4ebefde1"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:38.069198 15957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "55cd82ae10d04176bbcfa5347df04db7"
format_stamp: "Formatted at 2026-09-27 07:10:38 on dist-test-slave-zw7p"
server_key: "b51fcaaf4e40b57bf4258eea0ef1f17d"
server_key_iv: "ec92f07eff2f1e4fd5569b7a4ebefde1"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:38.070490 15957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:10:38.085152 15972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:38.085376 15957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:10:38.085505 15957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "55cd82ae10d04176bbcfa5347df04db7"
format_stamp: "Formatted at 2026-09-27 07:10:38 on dist-test-slave-zw7p"
server_key: "b51fcaaf4e40b57bf4258eea0ef1f17d"
server_key_iv: "ec92f07eff2f1e4fd5569b7a4ebefde1"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:38.085620 15957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:10:38.097422 15957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:10:38.099427 15957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:10:38.099582 15957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:10:38.104624 15957 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:42649
I20260927 07:10:38.104646 16024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:42649 every 8 connection(s)
I20260927 07:10:38.105136 15957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:10:38.105839 16025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:10:38.106693 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 15957
I20260927 07:10:38.106796 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:10:38.106972 4793 external_mini_cluster.cc:1428] Setting key 9f35e085646a9f51de0fa4c024dbdb57
I20260927 07:10:38.108386 16025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7: Bootstrap starting.
I20260927 07:10:38.109066 16025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7: Neither blocks nor log segments found. Creating new log.
I20260927 07:10:38.109390 16025 log.cc:824] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7: Log is configured to *not* fsync() on all Append() calls
I20260927 07:10:38.110065 16025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7: No bootstrap required, opened a new log
I20260927 07:10:38.112264 16025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55cd82ae10d04176bbcfa5347df04db7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42649 } }
I20260927 07:10:38.112407 16025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:10:38.112437 16025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cd82ae10d04176bbcfa5347df04db7, State: Initialized, Role: FOLLOWER
I20260927 07:10:38.112597 16025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55cd82ae10d04176bbcfa5347df04db7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42649 } }
I20260927 07:10:38.112663 16025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:10:38.112694 16025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:10:38.112754 16025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:10:38.113366 16025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55cd82ae10d04176bbcfa5347df04db7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42649 } }
I20260927 07:10:38.113473 16025 leader_election.cc:302] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55cd82ae10d04176bbcfa5347df04db7; no voters:
I20260927 07:10:38.113672 16025 leader_election.cc:288] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:10:38.113720 16030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:10:38.113950 16030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [term 1 LEADER]: Becoming Leader. State: Replica: 55cd82ae10d04176bbcfa5347df04db7, State: Running, Role: LEADER
I20260927 07:10:38.114039 16025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:10:38.114080 16030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55cd82ae10d04176bbcfa5347df04db7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42649 } }
I20260927 07:10:38.114573 16030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55cd82ae10d04176bbcfa5347df04db7. Latest consensus state: current_term: 1 leader_uuid: "55cd82ae10d04176bbcfa5347df04db7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55cd82ae10d04176bbcfa5347df04db7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42649 } } }
I20260927 07:10:38.114580 16031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55cd82ae10d04176bbcfa5347df04db7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55cd82ae10d04176bbcfa5347df04db7" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 42649 } } }
I20260927 07:10:38.114655 16030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [sys.catalog]: This master's current role is: LEADER
I20260927 07:10:38.114679 16031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7 [sys.catalog]: This master's current role is: LEADER
Sep 27 07:10:38 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:10:38.115063 16034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:10:38.115576 16034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:10:38.117126 16034 catalog_manager.cc:1347] Generated new cluster ID: 241f7d0b22bc46179d106d66607235be
I20260927 07:10:38.117165 16034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:10:38.119032 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:38.107550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.4.174.126:42649)
0927 07:10:38.107704 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:38.107715 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:38.107747 (+ 32us) server_negotiation.cc:407] Connection header received
0927 07:10:38.108358 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:38.108368 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:38.108435 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:38.108581 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:10:38.109321 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:38.110753 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:38.111823 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:38.112469 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:38.113115 (+ 646us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:38.115383 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:10:38.115387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:10:38.115387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:10:38.115398 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:10:38.117197 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:38.117564 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:38.117567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:38.117567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:38.117648 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:38.117843 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:38.117844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:38.117844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:38.118107 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:10:38.118225 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:38.118722 (+ 497us) server_negotiation.cc:299] Negotiation successful
0927 07:10:38.118832 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260927 07:10:38.119855 16025 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 07:10:38.130194 16034 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:10:38.131057 16034 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:10:38.136752 16034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7: Generated new TSK 0
I20260927 07:10:38.137002 16034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:10:38.820039 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42649
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:10:38.908525 16070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:10:38.908753 16070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:10:38.908778 16070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:10:38.910005 16070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:10:38.910058 16070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:10:38.910085 16070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:10:38.911684 16070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.174.84:42301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:10:38.911928 16070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:10:38.912163 16070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:10:38.917474 16075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:10:38.917825 16078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:10:38.918121 16070 server_base.cc:1034] running on GCE node
W20260927 07:10:38.920641 16076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:10:38.921558 16070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:10:38.921886 16070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:10:38.925103 16070 hybrid_clock.cc:648] HybridClock initialized: now 1790493038925004 us; error 111 us; skew 500 ppm
Sep 27 07:10:38 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493038, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:10:38.934551 16070 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:10:38.935079 16070 webserver.cc:460] Webserver started at http://127.4.174.65:45313/ using document root <none> and password file <none>
I20260927 07:10:38.935315 16070 fs_manager.cc:359] Metadata directory not provided
I20260927 07:10:38.935360 16070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:10:38.935467 16070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:10:38 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493038, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:10:38.949687 16070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "658264ca1a234ecd954361319c849375"
format_stamp: "Formatted at 2026-09-27 07:10:38 on dist-test-slave-zw7p"
server_key: "07461504351222854f285ce593c2f092"
server_key_iv: "356c580df6f7fd496da3224833c37204"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:38.950212 16070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "658264ca1a234ecd954361319c849375"
format_stamp: "Formatted at 2026-09-27 07:10:38 on dist-test-slave-zw7p"
server_key: "07461504351222854f285ce593c2f092"
server_key_iv: "356c580df6f7fd496da3224833c37204"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:38.952118 16070 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260927 07:10:38.967509 16085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:38.967808 16070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:10:38.967923 16070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "658264ca1a234ecd954361319c849375"
format_stamp: "Formatted at 2026-09-27 07:10:38 on dist-test-slave-zw7p"
server_key: "07461504351222854f285ce593c2f092"
server_key_iv: "356c580df6f7fd496da3224833c37204"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:38.968032 16070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:10:38.984941 16070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:10:38.987406 16070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:10:38.987558 16070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:10:38.987819 16070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:10:38.988178 16070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:10:38.988234 16070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:38.988265 16070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:10:38.988287 16070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:38.994793 16070 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:33157
I20260927 07:10:38.995179 16198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:33157 every 8 connection(s)
I20260927 07:10:38.995280 16070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:10:38.997125 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 16070
I20260927 07:10:38.997231 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:10:38.997432 4793 external_mini_cluster.cc:1428] Setting key 2d6c3f2e1f3808af650276cfb9e8dab8
Sep 27 07:10:38 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493038, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
I20260927 07:10:39.008360 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:38.996742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:37303 (local address 127.4.174.126:42649)
0927 07:10:38.996873 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:38.996876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:38.997791 (+ 915us) server_negotiation.cc:407] Connection header received
0927 07:10:38.997814 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:38.997815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:38.997828 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:38.997887 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:10:38.998696 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:38.999406 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.000890 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.001096 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.001176 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.005600 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:10:39.005602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:10:39.005602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:10:39.005610 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:10:39.007006 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:39.007400 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:39.007402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:39.007402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:39.007456 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:39.007712 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:39.007713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:39.007713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:39.007830 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:10:39.007889 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.008162 (+ 273us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.008253 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260927 07:10:39.008490 16202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:38.996440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42649 (local address 127.4.174.65:37303)
0927 07:10:38.996949 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:38.997005 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 07:10:38.997648 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:10:38.998012 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:10:38.998017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:10:38.998259 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:10:38.998590 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:38.998598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:38.999510 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:38.999511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.000780 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.000785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.001211 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.001212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.001359 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.001924 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:10:39.001934 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:10:39.005486 (+ 3552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:10:39.007083 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:10:39.007091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:10:39.007094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:10:39.007333 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:10:39.007510 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:10:39.007511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:10:39.007512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:10:39.007647 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:10:39.007883 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:10:39.007889 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:10:39.008035 (+ 146us) client_negotiation.cc:769] Sending connection context
0927 07:10:39.008125 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 07:10:39.008272 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260927 07:10:39.009027 16200 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42649
I20260927 07:10:39.009119 16200 heartbeater.cc:464] Registering TS with master...
I20260927 07:10:39.009301 16200 heartbeater.cc:511] Master 127.4.174.126:42649 requested a full tablet report, sending...
I20260927 07:10:39.009891 15989 ts_manager.cc:194] Registered new tserver with Master: 658264ca1a234ecd954361319c849375 (127.4.174.65:33157)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
I20260927 07:10:39.010838 15989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:37303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:10:39.025684 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42649
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:10:39.141572 16207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:10:39.141875 16207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:10:39.141921 16207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:10:39.143757 16207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:10:39.143821 16207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:10:39.143862 16207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:10:39.146077 16207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.174.84:42301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:10:39.146327 16207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:10:39.146667 16207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:10:39.152865 16214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:10:39.152912 16217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:10:39.153249 16207 server_base.cc:1034] running on GCE node
W20260927 07:10:39.153328 16215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:10:39.153549 16207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:10:39.153828 16207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:10:39.154989 16207 hybrid_clock.cc:648] HybridClock initialized: now 1790493039154967 us; error 35 us; skew 500 ppm
Sep 27 07:10:39 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:10:39.158301 16207 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:10:39.158913 16207 webserver.cc:460] Webserver started at http://127.4.174.66:40633/ using document root <none> and password file <none>
I20260927 07:10:39.159170 16207 fs_manager.cc:359] Metadata directory not provided
I20260927 07:10:39.159235 16207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:10:39.159366 16207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:10:39 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:10:39.175360 16207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "b662074824aa4786a17e78a2bee33b4f"
format_stamp: "Formatted at 2026-09-27 07:10:39 on dist-test-slave-zw7p"
server_key: "c0cd7bffdc4e5675851968d192c3f9af"
server_key_iv: "3b22abf34af1baf440e97f57a54976c5"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:39.175971 16207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "b662074824aa4786a17e78a2bee33b4f"
format_stamp: "Formatted at 2026-09-27 07:10:39 on dist-test-slave-zw7p"
server_key: "c0cd7bffdc4e5675851968d192c3f9af"
server_key_iv: "3b22abf34af1baf440e97f57a54976c5"
server_key_version: "kuduclusterkey@0"
Sep 27 07:10:39 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790493039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:10:39.183012 16207 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
I20260927 07:10:39.199884 16224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:39.200109 16207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:10:39.200191 16207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "b662074824aa4786a17e78a2bee33b4f"
format_stamp: "Formatted at 2026-09-27 07:10:39 on dist-test-slave-zw7p"
server_key: "c0cd7bffdc4e5675851968d192c3f9af"
server_key_iv: "3b22abf34af1baf440e97f57a54976c5"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:39.200280 16207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:10:39.211766 16207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:10:39.215754 16207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:10:39.215941 16207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:10:39.216173 16207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:10:39.216573 16207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:10:39.216651 16207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:39.216711 16207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:10:39.216754 16207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:39.222957 16207 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:41247
I20260927 07:10:39.223507 16207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:10:39.227383 16337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:41247 every 8 connection(s)
I20260927 07:10:39.233332 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 16207
I20260927 07:10:39.233449 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:10:39.233659 4793 external_mini_cluster.cc:1428] Setting key eae751d5f6647c5faf3342fbb8e9d385
Sep 27 07:10:39 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:10:39.244863 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.228373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:43215 (local address 127.4.174.126:42649)
0927 07:10:39.228419 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.228421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.234633 (+ 6212us) server_negotiation.cc:407] Connection header received
0927 07:10:39.235343 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.235345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.235358 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.235386 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:10:39.236251 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.237067 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.238717 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.239041 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.239080 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.241638 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:10:39.241639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:10:39.241640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:10:39.241647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:10:39.243332 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:39.243800 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:39.243801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:39.243801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:39.243881 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:39.244142 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:39.244142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:39.244143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:39.244300 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:10:39.244371 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.244665 (+ 294us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.244759 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:10:39.245021 16340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.228087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42649 (local address 127.4.174.66:43215)
0927 07:10:39.234443 (+ 6356us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:39.234509 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0927 07:10:39.235220 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:10:39.235580 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:10:39.235585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:10:39.235860 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:10:39.236147 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.236154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.237181 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.237183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.238607 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.238610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.239166 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.239166 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.239313 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.239907 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:10:39.239918 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:10:39.241533 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:10:39.243417 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:10:39.243424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:10:39.243427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:10:39.243708 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:10:39.243953 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:10:39.243954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:10:39.243954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:10:39.244080 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:10:39.244362 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:10:39.244367 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:10:39.244524 (+ 157us) client_negotiation.cc:769] Sending connection context
0927 07:10:39.244627 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 07:10:39.244778 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6296,"thread_start_us":79,"threads_started":1}
I20260927 07:10:39.245446 16338 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42649
I20260927 07:10:39.245543 16338 heartbeater.cc:464] Registering TS with master...
I20260927 07:10:39.245739 16338 heartbeater.cc:511] Master 127.4.174.126:42649 requested a full tablet report, sending...
I20260927 07:10:39.246068 15989 ts_manager.cc:194] Registered new tserver with Master: b662074824aa4786a17e78a2bee33b4f (127.4.174.66:41247)
I20260927 07:10:39.246603 15989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:43215
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:10:39.278141 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42649
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:10:39.401773 16344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:10:39.402074 16344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:10:39.402113 16344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:10:39.404011 16344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:10:39.404153 16344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:10:39.404210 16344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:10:39.406657 16344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.174.84:42301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:42649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:10:39.406968 16344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:10:39.407289 16344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:10:39.412695 16350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:10:39.412695 16349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:10:39.412760 16352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:10:39.413394 16344 server_base.cc:1034] running on GCE node
I20260927 07:10:39.413604 16344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:10:39.413842 16344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:10:39.414992 16344 hybrid_clock.cc:648] HybridClock initialized: now 1790493039414977 us; error 32 us; skew 500 ppm
Sep 27 07:10:39 dist-test-slave-zw7p krb5kdc[15513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:10:39.417784 16344 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:10:39.418355 16344 webserver.cc:460] Webserver started at http://127.4.174.67:40051/ using document root <none> and password file <none>
I20260927 07:10:39.418699 16344 fs_manager.cc:359] Metadata directory not provided
I20260927 07:10:39.418771 16344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:10:39.418912 16344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:10:39 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:10:39.434790 16344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "064d9b56ba3e4a0094152e7ecfacb864"
format_stamp: "Formatted at 2026-09-27 07:10:39 on dist-test-slave-zw7p"
server_key: "33a6053717f04a48821ca8933322ecf3"
server_key_iv: "45f418b704b7a1b9f7bbb91431734ffd"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:39.435263 16344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "064d9b56ba3e4a0094152e7ecfacb864"
format_stamp: "Formatted at 2026-09-27 07:10:39 on dist-test-slave-zw7p"
server_key: "33a6053717f04a48821ca8933322ecf3"
server_key_iv: "45f418b704b7a1b9f7bbb91431734ffd"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:39.442780 16344 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260927 07:10:39.456688 16359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:39.456861 16344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:10:39.456944 16344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "064d9b56ba3e4a0094152e7ecfacb864"
format_stamp: "Formatted at 2026-09-27 07:10:39 on dist-test-slave-zw7p"
server_key: "33a6053717f04a48821ca8933322ecf3"
server_key_iv: "45f418b704b7a1b9f7bbb91431734ffd"
server_key_version: "kuduclusterkey@0"
I20260927 07:10:39.457036 16344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:10:39.503245 16344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:10:39.505316 16344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:10:39.505499 16344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:10:39.505712 16344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:10:39.506067 16344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:10:39.506105 16344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:39.506134 16344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:10:39.506155 16344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:10:39.512056 16344 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:45727
I20260927 07:10:39.512513 16344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:10:39.515430 16472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:45727 every 8 connection(s)
Sep 27 07:10:39 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:10:39.520875 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 16344
I20260927 07:10:39.520958 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:10:39.521142 4793 external_mini_cluster.cc:1428] Setting key 198c2f1d3dda6062a83682b91908c6d9
I20260927 07:10:39.525418 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.515017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:57295 (local address 127.4.174.126:42649)
0927 07:10:39.515063 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.515066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.515740 (+ 674us) server_negotiation.cc:407] Connection header received
0927 07:10:39.516430 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.516431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.516444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.516502 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:10:39.517008 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.517777 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.518871 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.519184 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.519252 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.521343 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:10:39.521345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:10:39.521345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:10:39.521353 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:10:39.523022 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:39.524440 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:39.524442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:39.524442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:39.524501 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:39.524712 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:39.524713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:39.524713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:39.524828 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:10:39.524885 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.525244 (+ 359us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.525328 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:10:39.525564 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.515024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:42649 (local address 127.4.174.67:57295)
0927 07:10:39.515599 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:39.515642 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:10:39.516293 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:10:39.516559 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:10:39.516563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:10:39.516747 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:10:39.516941 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.516945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.517862 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.517863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.518805 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.518808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.519266 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.519268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.519368 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.519837 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:10:39.519845 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:10:39.521259 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:10:39.524137 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:10:39.524143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:10:39.524145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:10:39.524364 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:10:39.524556 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:10:39.524557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:10:39.524557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:10:39.524655 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:10:39.524979 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:10:39.524985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:10:39.525106 (+ 121us) client_negotiation.cc:769] Sending connection context
0927 07:10:39.525202 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 07:10:39.525343 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":85,"threads_started":1}
I20260927 07:10:39.526149 16473 heartbeater.cc:347] Connected to a master server at 127.4.174.126:42649
I20260927 07:10:39.526247 16473 heartbeater.cc:464] Registering TS with master...
I20260927 07:10:39.526424 16473 heartbeater.cc:511] Master 127.4.174.126:42649 requested a full tablet report, sending...
I20260927 07:10:39.526804 15989 ts_manager.cc:194] Registered new tserver with Master: 064d9b56ba3e4a0094152e7ecfacb864 (127.4.174.67:45727)
I20260927 07:10:39.527307 15989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:57295
I20260927 07:10:39.533069 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:10:39.541155 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.534587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.4.174.126:42649)
0927 07:10:39.534642 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.534644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.534768 (+ 124us) server_negotiation.cc:407] Connection header received
0927 07:10:39.534836 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.534838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.534849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.534907 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:10:39.535586 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.536125 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.536869 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.537066 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.537127 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.538399 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:10:39.538400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:10:39.538400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:10:39.538408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:10:39.539854 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:39.540233 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:39.540235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:39.540235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:39.540304 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:10:39.540536 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:10:39.540537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:10:39.540538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:10:39.540694 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:10:39.540762 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.540964 (+ 202us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.541042 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:10:39.542970 15989 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:10:39.545157 15989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:10:39.553239 16487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.547912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:41247 (local address 127.0.0.1:50916)
0927 07:10:39.548964 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:39.548974 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:10:39.549021 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:10:39.549292 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:10:39.549293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:10:39.549294 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:10:39.549335 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.549337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.550500 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.550501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.551881 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.551884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.552926 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.552927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.553006 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.553020 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 07:10:39.553091 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 07:10:39.553159 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"mutex_wait_us":91,"thread_start_us":63,"threads_started":1}
I20260927 07:10:39.553640 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.548222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50566 (local address 127.4.174.65:33157)
0927 07:10:39.548657 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.548660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.548669 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:10:39.548697 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.548699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.548713 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.548781 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:10:39.548961 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.550054 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.551990 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.553082 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.553142 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.553196 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.553518 (+ 322us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.553560 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":52,"threads_started":1}
I20260927 07:10:39.553690 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.548042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.4.174.66:41247)
0927 07:10:39.548909 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.548914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.549074 (+ 160us) server_negotiation.cc:407] Connection header received
0927 07:10:39.549115 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.549117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.549146 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.549218 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:10:39.549411 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.550416 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.552002 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.552746 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.552811 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.552889 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.553571 (+ 682us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.553610 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":89,"threads_started":1}
I20260927 07:10:39.553455 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.547653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:33157 (local address 127.0.0.1:50566)
0927 07:10:39.547973 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:39.548464 (+ 491us) client_negotiation.cc:174] Beginning negotiation
0927 07:10:39.548525 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:10:39.548796 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:10:39.548799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:10:39.548801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:10:39.548872 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.548874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.550159 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.550160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.551881 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.551884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.553228 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.553229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.553295 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.553299 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:10:39.553354 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 07:10:39.553395 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260927 07:10:39.554692 16272 tablet_service.cc:1467] Processing CreateTablet for tablet 80e95dd153bf42d68b9b3daa976d9c3e (DEFAULT_TABLE table=test-table [id=122b0fa6bde2451b85d77ef963e25be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:10:39.555102 16272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e95dd153bf42d68b9b3daa976d9c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:10:39.557420 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.548095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:45727 (local address 127.0.0.1:46282)
0927 07:10:39.551019 (+ 2924us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:39.554286 (+ 3267us) client_negotiation.cc:174] Beginning negotiation
0927 07:10:39.554382 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:10:39.554596 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:10:39.554596 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:10:39.554598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:10:39.554657 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.554659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.555654 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.555655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.556467 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.556469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.557175 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.557176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.557237 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.557242 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:10:39.557303 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 07:10:39.557342 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2543,"thread_start_us":59,"threads_started":1}
I20260927 07:10:39.557438 16486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.547758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.4.174.67:45727)
0927 07:10:39.548401 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.548406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.554349 (+ 5943us) server_negotiation.cc:407] Connection header received
0927 07:10:39.554478 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.554481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.554497 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.554595 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:10:39.554718 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.555577 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.556573 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.557092 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.557167 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.557243 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.557317 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.557358 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":77,"threads_started":1}
I20260927 07:10:39.558354 16404 tablet_service.cc:1467] Processing CreateTablet for tablet 80e95dd153bf42d68b9b3daa976d9c3e (DEFAULT_TABLE table=test-table [id=122b0fa6bde2451b85d77ef963e25be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:10:39.558735 16404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e95dd153bf42d68b9b3daa976d9c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:10:39.558737 16491 tablet_bootstrap.cc:492] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f: Bootstrap starting.
I20260927 07:10:39.559070 16127 tablet_service.cc:1467] Processing CreateTablet for tablet 80e95dd153bf42d68b9b3daa976d9c3e (DEFAULT_TABLE table=test-table [id=122b0fa6bde2451b85d77ef963e25be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:10:39.559412 16127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e95dd153bf42d68b9b3daa976d9c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:10:39.559983 16491 tablet_bootstrap.cc:654] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f: Neither blocks nor log segments found. Creating new log.
I20260927 07:10:39.560392 16491 log.cc:824] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:10:39.561220 16491 tablet_bootstrap.cc:492] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f: No bootstrap required, opened a new log
I20260927 07:10:39.561293 16491 ts_tablet_manager.cc:1397] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:10:39.563251 16494 tablet_bootstrap.cc:492] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Bootstrap starting.
I20260927 07:10:39.563411 16491 raft_consensus.cc:348] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.563581 16491 raft_consensus.cc:374] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:10:39.563618 16491 raft_consensus.cc:729] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b662074824aa4786a17e78a2bee33b4f, State: Initialized, Role: FOLLOWER
I20260927 07:10:39.563741 16491 consensus_queue.cc:260] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.564023 16491 ts_tablet_manager.cc:1428] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:10:39.564164 16338 heartbeater.cc:503] Master 127.4.174.126:42649 was elected leader, sending a full tablet report...
I20260927 07:10:39.564239 16494 tablet_bootstrap.cc:654] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Neither blocks nor log segments found. Creating new log.
I20260927 07:10:39.564558 16494 log.cc:824] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Log is configured to *not* fsync() on all Append() calls
I20260927 07:10:39.565313 16494 tablet_bootstrap.cc:492] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: No bootstrap required, opened a new log
I20260927 07:10:39.565376 16494 ts_tablet_manager.cc:1397] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:10:39.567480 16494 raft_consensus.cc:348] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.567680 16494 raft_consensus.cc:374] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:10:39.567724 16494 raft_consensus.cc:729] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064d9b56ba3e4a0094152e7ecfacb864, State: Initialized, Role: FOLLOWER
I20260927 07:10:39.567852 16494 consensus_queue.cc:260] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.568156 16494 ts_tablet_manager.cc:1428] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:10:39.568131 16493 tablet_bootstrap.cc:492] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375: Bootstrap starting.
I20260927 07:10:39.568692 16473 heartbeater.cc:503] Master 127.4.174.126:42649 was elected leader, sending a full tablet report...
I20260927 07:10:39.569056 16493 tablet_bootstrap.cc:654] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375: Neither blocks nor log segments found. Creating new log.
I20260927 07:10:39.569476 16493 log.cc:824] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375: Log is configured to *not* fsync() on all Append() calls
I20260927 07:10:39.570297 16493 tablet_bootstrap.cc:492] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375: No bootstrap required, opened a new log
I20260927 07:10:39.570370 16493 ts_tablet_manager.cc:1397] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:10:39.572993 16493 raft_consensus.cc:348] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.573145 16493 raft_consensus.cc:374] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:10:39.573181 16493 raft_consensus.cc:729] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658264ca1a234ecd954361319c849375, State: Initialized, Role: FOLLOWER
I20260927 07:10:39.573342 16493 consensus_queue.cc:260] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.573895 16493 ts_tablet_manager.cc:1428] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:10:39.574045 16200 heartbeater.cc:503] Master 127.4.174.126:42649 was elected leader, sending a full tablet report...
Sep 27 07:10:39 dist-test-slave-zw7p krb5kdc[15513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
W20260927 07:10:39.728955 16339 tablet.cc:2369] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:10:39.746378 16201 tablet.cc:2369] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:10:39.764633 16474 tablet.cc:2369] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:10:39.822106 16497 raft_consensus.cc:482] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:10:39.822286 16497 raft_consensus.cc:504] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.822782 16497 leader_election.cc:288] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658264ca1a234ecd954361319c849375 (127.4.174.65:33157), b662074824aa4786a17e78a2bee33b4f (127.4.174.66:41247)
I20260927 07:10:39.827068 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.822983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:33157 (local address 127.4.174.67:57169)
0927 07:10:39.823275 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:39.823287 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:10:39.823358 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:10:39.823633 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:10:39.823634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:10:39.823635 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:10:39.823711 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.823713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.824721 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.824723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.825946 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.825949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.826808 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.826809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.826882 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.826887 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:10:39.826942 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 07:10:39.826987 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
I20260927 07:10:39.827597 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.823085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:57169 (local address 127.4.174.65:33157)
0927 07:10:39.823146 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.823148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.823484 (+ 336us) server_negotiation.cc:407] Connection header received
0927 07:10:39.823517 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.823518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.823533 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.823581 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:10:39.823865 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.824612 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.826078 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.826686 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.826761 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.826772 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.827475 (+ 703us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.827525 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:10:39.827863 16153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e95dd153bf42d68b9b3daa976d9c3e" candidate_uuid: "064d9b56ba3e4a0094152e7ecfacb864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658264ca1a234ecd954361319c849375" is_pre_election: true
I20260927 07:10:39.828016 16153 raft_consensus.cc:2454] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 064d9b56ba3e4a0094152e7ecfacb864 in term 0.
I20260927 07:10:39.828274 16363 leader_election.cc:302] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064d9b56ba3e4a0094152e7ecfacb864, 658264ca1a234ecd954361319c849375; no voters:
I20260927 07:10:39.828636 16497 raft_consensus.cc:2781] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:10:39.828701 16497 raft_consensus.cc:482] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:10:39.828725 16497 raft_consensus.cc:3037] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:10:39.829675 16497 raft_consensus.cc:504] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.829813 16497 leader_election.cc:288] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [CANDIDATE]: Term 1 election: Requested vote from peers 658264ca1a234ecd954361319c849375 (127.4.174.65:33157), b662074824aa4786a17e78a2bee33b4f (127.4.174.66:41247)
I20260927 07:10:39.830010 16153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e95dd153bf42d68b9b3daa976d9c3e" candidate_uuid: "064d9b56ba3e4a0094152e7ecfacb864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658264ca1a234ecd954361319c849375"
I20260927 07:10:39.830106 16153 raft_consensus.cc:3037] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:10:39.831046 16153 raft_consensus.cc:2454] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 064d9b56ba3e4a0094152e7ecfacb864 in term 1.
I20260927 07:10:39.831243 16363 leader_election.cc:302] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064d9b56ba3e4a0094152e7ecfacb864, 658264ca1a234ecd954361319c849375; no voters:
I20260927 07:10:39.831418 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.823446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:47539 (local address 127.4.174.66:41247)
0927 07:10:39.823504 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.823506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.823514 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:10:39.823540 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.823541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.823551 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.823599 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:10:39.823819 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.824519 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.830380 (+ 5861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.830978 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.831030 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.831039 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.831298 (+ 259us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.831346 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 07:10:39.831472 16497 raft_consensus.cc:2781] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:10:39.831655 16497 raft_consensus.cc:686] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 1 LEADER]: Becoming Leader. State: Replica: 064d9b56ba3e4a0094152e7ecfacb864, State: Running, Role: LEADER
I20260927 07:10:39.831770 16497 consensus_queue.cc:237] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } }
I20260927 07:10:39.831643 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.823092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:41247 (local address 127.4.174.67:47539)
0927 07:10:39.823293 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:39.823304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:10:39.823365 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:10:39.823649 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:10:39.823650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:10:39.823652 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:10:39.823729 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.823731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.827204 (+ 3473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.827205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.828315 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:10:39.828318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.831121 (+ 2803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.831122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:10:39.831192 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.831196 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:10:39.831502 (+ 306us) client_negotiation.cc:240] Negotiation successful
0927 07:10:39.831552 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":94,"threads_started":1}
I20260927 07:10:39.832607 15989 catalog_manager.cc:5495] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 reported cstate change: term changed from 0 to 1, leader changed from <none> to 064d9b56ba3e4a0094152e7ecfacb864 (127.4.174.67). New cstate: current_term: 1 leader_uuid: "064d9b56ba3e4a0094152e7ecfacb864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "064d9b56ba3e4a0094152e7ecfacb864" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 45727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 } health_report { overall_health: UNKNOWN } } }
I20260927 07:10:39.833065 16292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e95dd153bf42d68b9b3daa976d9c3e" candidate_uuid: "064d9b56ba3e4a0094152e7ecfacb864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b662074824aa4786a17e78a2bee33b4f" is_pre_election: true
I20260927 07:10:39.833072 16291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e95dd153bf42d68b9b3daa976d9c3e" candidate_uuid: "064d9b56ba3e4a0094152e7ecfacb864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b662074824aa4786a17e78a2bee33b4f"
I20260927 07:10:39.833235 16292 raft_consensus.cc:2454] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 064d9b56ba3e4a0094152e7ecfacb864 in term 0.
I20260927 07:10:39.843024 16486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.839840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.4.174.67:45727)
0927 07:10:39.839882 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:10:39.839884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:10:39.839910 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:10:39.839996 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:10:39.839997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:10:39.840016 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:10:39.840091 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:10:39.840221 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.840940 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.842076 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:10:39.842399 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:10:39.842470 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:10:39.842669 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:10:39.842810 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:10:39.842899 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 07:10:39.842923 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 07:10:39.842944 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:10:39.844897 16291 raft_consensus.cc:3037] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:10:39.844995 16153 raft_consensus.cc:1261] T 80e95dd153bf42d68b9b3daa976d9c3e P 658264ca1a234ecd954361319c849375 [term 1 FOLLOWER]: Refusing update from remote peer 064d9b56ba3e4a0094152e7ecfacb864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:10:39.845276 16497 consensus_queue.cc:1035] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658264ca1a234ecd954361319c849375" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 33157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:10:39.845880 16291 raft_consensus.cc:1261] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f [term 1 FOLLOWER]: Refusing update from remote peer 064d9b56ba3e4a0094152e7ecfacb864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:10:39.846288 16497 consensus_queue.cc:1035] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b662074824aa4786a17e78a2bee33b4f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:10:39.851596 16504 mvcc.cc:205] Tried to move back new op lower bound from 7333859491202342912 to 7333859491151441920. Current Snapshot: MvccSnapshot[applied={T|T < 7333859491202342912}]
I20260927 07:10:39.852914 16505 mvcc.cc:205] Tried to move back new op lower bound from 7333859491202342912 to 7333859491151441920. Current Snapshot: MvccSnapshot[applied={T|T < 7333859491202342912}]
I20260927 07:10:39.854521 16507 mvcc.cc:205] Tried to move back new op lower bound from 7333859491202342912 to 7333859491151441920. Current Snapshot: MvccSnapshot[applied={T|T < 7333859491202342912}]
I20260927 07:10:39.856783 15989 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:10:39.856899 15989 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:10:39.857614 15989 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55cd82ae10d04176bbcfa5347df04db7: Sending DeleteTablet for 3 replicas of tablet 80e95dd153bf42d68b9b3daa976d9c3e
I20260927 07:10:39.857944 16404 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e95dd153bf42d68b9b3daa976d9c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:10:39 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:46282
I20260927 07:10:39.858395 16127 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e95dd153bf42d68b9b3daa976d9c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:10:39 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50566
I20260927 07:10:39.858642 16272 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e95dd153bf42d68b9b3daa976d9c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:10:39 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50916
I20260927 07:10:39.858757 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 16070
I20260927 07:10:39.861331 16514 tablet_replica.cc:330] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: stopping tablet replica
I20260927 07:10:39.861444 16514 raft_consensus.cc:2229] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:10:39.861543 16514 raft_consensus.cc:2258] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:10:39.862095 16514 ts_tablet_manager.cc:1905] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:10:39.866575 16514 ts_tablet_manager.cc:1918] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:10:39.866657 16514 log.cc:1196] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/80e95dd153bf42d68b9b3daa976d9c3e
I20260927 07:10:39.866918 16514 ts_tablet_manager.cc:1939] T 80e95dd153bf42d68b9b3daa976d9c3e P 064d9b56ba3e4a0094152e7ecfacb864: Deleting consensus metadata
I20260927 07:10:39.867324 15974 catalog_manager.cc:4841] TS 064d9b56ba3e4a0094152e7ecfacb864 (127.4.174.67:45727): tablet 80e95dd153bf42d68b9b3daa976d9c3e (table test-table [id=122b0fa6bde2451b85d77ef963e25be7]) successfully deleted
W20260927 07:10:39.867614 15976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:33157 (error 108)
I20260927 07:10:39.867854 16516 tablet_replica.cc:330] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f: stopping tablet replica
I20260927 07:10:39.868044 16516 raft_consensus.cc:2229] T 80e95dd153bf42d68b9b3daa976d9c3e P b662074824aa4786a17e78a2bee33b4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:10:39.868120 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 16207
I20260927 07:10:39.880040 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 16344
I20260927 07:10:39.880030 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:10:39.879884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:33157 (local address 127.0.0.1:50582)
0927 07:10:39.879935 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 07:10:39.879981 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:33157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260927 07:10:39.880189 15976 catalog_manager.cc:4584] TS 658264ca1a234ecd954361319c849375 (127.4.174.65:33157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 80e95dd153bf42d68b9b3daa976d9c3e: Network error: Client connection negotiation failed: client connection to 127.4.174.65:33157: connect: Connection refused (error 111)
I20260927 07:10:39.886199 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 15957
I20260927 07:10:39.890981 4793 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 07:10:40.225301 4793 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 07:10:40.225400 4793 mini_ranger.cc:66] Stopping Ranger...
I20260927 07:10:40.559780 4793 mini_ranger.cc:68] Stopped Ranger
2026-09-27 07:10:40.559 UTC [15543] LOG: received smart shutdown request
2026-09-27 07:10:40.563 UTC [15543] LOG: background worker "logical replication launcher" (PID 15552) exited with exit code 1
2026-09-27 07:10:40.563 UTC [15547] LOG: shutting down
2026-09-27 07:10:40.720 UTC [15543] LOG: database system is shut down
2026-09-27T07:10:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44987 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:10:40 dist-test-slave-zw7p krb5kdc[16527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:10:40 dist-test-slave-zw7p krb5kdc[16527](info): set up 2 sockets
Sep 27 07:10:40 dist-test-slave-zw7p krb5kdc[16527](info): commencing operation
krb5kdc: starting...
W20260927 07:10:41.750407 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:10:41 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:10:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:10:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.174.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.174.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/rangeradmin_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/rangeradmin_127.4.174.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.174.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.174.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/rangerlookup_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/rangerlookup_127.4.174.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.174.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.174.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/HTTP_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/HTTP_127.4.174.84@KRBTEST.COM.keytab.
I20260927 07:10:41.907389 4793 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIteyrz/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/postgres -l logfile start
2026-09-27 07:10:44.237 UTC [16557] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 07:10:44.237 UTC [16557] LOG: listening on IPv4 address "127.4.174.84", port 42763
2026-09-27 07:10:44.240 UTC [16557] LOG: listening on Unix socket "/tmp/.s.PGSQL.42763"
2026-09-27 07:10:44.259 UTC [16560] LOG: database system was shut down at 2026-09-27 07:10:42 UTC
2026-09-27 07:10:44.262 UTC [16557] LOG: database system is ready to accept connections
I20260927 07:10:45.126127 4793 mini_postgres.cc:96] Postgres bound to 42763
2026-09-27 07:10:45.131 UTC [16568] FATAL: database "slave" does not exist
127.4.174.84:42763 - accepting connections
I20260927 07:10:45.132287 4793 mini_ranger.cc:161] Starting Ranger...
I20260927 07:10:45.145727 4793 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 07:10:45.564985 4793 mini_ranger.cc:87] Created ranger Postgres database
I20260927 07:10:45.565050 4793 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-admin
2026-09-27 07:10:45,832 [I] DB FLAVOR :POSTGRES
2026-09-27 07:10:45,832 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 07:10:45,832 [I] Checking connection..
2026-09-27 07:10:45,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 07:10:46,090 [I] Checking connection passed.
2026-09-27 07:10:46,091 [I] --------- Verifying version history table ---------
2026-09-27 07:10:46,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:10:46,323 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 07:10:46,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:10:46,573 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 07:10:46,573 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 07:10:46,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 07:10:46.814 UTC [16632] WARNING: there is no transaction in progress
2026-09-27 07:10:46,819 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 07:10:46,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:10:47,066 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 07:10:47,066 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 07:10:47,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 07:10:47,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 07:10:47,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zw7p.c.gcp-upstream.internal','N') ;"
2026-09-27 07:10:47,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 07:10:48,061 [I] Table x_portal_user does not exist in database ranger
2026-09-27 07:10:48,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 07:10:48,292 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 07:10:48,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 07:10:48,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 07:10:48,782 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 07:10:48,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 07:10:49.720 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.736 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.752 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.768 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.783 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.800 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.856 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.862 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.877 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.887 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.898 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.910 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.920 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.927 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.933 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:49.938 UTC [16740] WARNING: there is no transaction in progress
2026-09-27 07:10:50,569 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 07:10:50,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 07:10:50,816 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 07:10:50,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 07:10:51,064 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 07:10:51,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 07:10:51,321 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 07:10:51,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 07:10:51,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 07:10:51,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zw7p.c.gcp-upstream.internal';"
2026-09-27 07:10:52,036 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 07:10:52,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 07:10:52,279 [I] DB_PATCHES have already been applied
I20260927 07:10:52.284577 4793 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIteyrz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 07:10:52.284644 4793 mini_ranger.cc:193] Using host: 127.4.174.84
I20260927 07:10:52.287528 4793 mini_ranger.cc:227] Ranger admin URL: http://127.4.174.84:39343
Sep 27, 2026 7:10:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 7:10:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 7:10:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 7:10:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 7:10:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 7:10:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 7:10:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 07:10:53.284624 4793 mini_ranger.cc:160] Time spent starting Ranger: real 8.152s user 0.001s sys 0.006s
Sep 27, 2026 7:11:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 7:11:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 7:11:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 7:11:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 7:11:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 07:11:14.377668 4793 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.174.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.174.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/rangerkms_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.174.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/rangerkms_127.4.174.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.174.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.174.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.174.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/HTTP_127.4.174.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.174.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/HTTP_127.4.174.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 07:11:14 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493074, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 07:11:14.460340 4793 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 07:11:14.469663 4793 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 07:11:15.330482 4793 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 07:11:15.330582 4793 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms
I20260927 07:11:15.330603 4793 mini_ranger_kms.cc:227] Using postgres at 127.4.174.84:42763
2026-09-27 07:11:15,481 [I] DB FLAVOR :POSTGRES
2026-09-27 07:11:15,481 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 07:11:15,481 [I] Checking connection
2026-09-27 07:11:15,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 07:11:15,714 [I] connection success
2026-09-27 07:11:15,714 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 07:11:15,715 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 07:11:15,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 07:11:15,956 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 07:11:15,956 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 07:11:15,956 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 07:11:15,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.174.84:42763/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 07:11:16,222 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 07:11:16.644205 4793 mini_ranger_kms.cc:310] Created kms service
I20260927 07:11:16.812283 4793 mini_ranger_kms.cc:326] Created kudu user
I20260927 07:11:16.856839 4793 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 07:11:16 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493074, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:17.317003 4793 mini_ranger_kms.cc:384] Added ranger policy
I20260927 07:11:17.317134 4793 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIteyrz/build/release/bin/postgresql.jar:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIteyrz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 07:11:17.317158 4793 mini_ranger_kms.cc:240] Using host: 127.4.174.84
I20260927 07:11:17.320215 4793 mini_ranger_kms.cc:276] Ranger KMS PID: 16920
I20260927 07:11:17.320291 4793 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.174.84:42671
Sep 27, 2026 7:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 7:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 7:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 7:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 7:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 7:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 07:11:19.313905 4793 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.854s user 0.003s sys 0.132s
I20260927 07:11:19.314006 4793 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 07:11:19.314066 4793 mini_ranger_kms.cc:401] 127.4.174.84:42671/kms/v1/keys
Sep 27 07:11:19 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790493079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.174.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 07:11:19 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.174.84@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
Sep 27 07:11:20 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.174.84@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
Sep 27 07:11:22 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:11:22.497186 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40767
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40767
--ranger_config_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42671/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:22.576732 16971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:22.576936 16971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:22.576963 16971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:22.578110 16971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:22.578151 16971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:22.578171 16971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:22.578186 16971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:22.578200 16971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:22.579770 16971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.174.84:42671/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40767
--ranger_config_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:22.580009 16971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:22.580214 16971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:22.585165 16979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:22.585206 16977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:22.585214 16976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:22.585237 16971 server_base.cc:1034] running on GCE node
I20260927 07:11:22.585551 16971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:22.585820 16971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:22.586961 16971 hybrid_clock.cc:648] HybridClock initialized: now 1790493082586938 us; error 39 us; skew 500 ppm
Sep 27 07:11:22 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493082, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:22.594023 16971 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:11:22.594544 16971 webserver.cc:460] Webserver started at http://127.4.174.126:40781/ using document root <none> and password file <none>
I20260927 07:11:22.594830 16971 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:22.594880 16971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:22.595022 16971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:11:22 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493082, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:22.651872 16971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "c86cd9c448e74cf5a5b84c914e9c2664"
format_stamp: "Formatted at 2026-09-27 07:11:22 on dist-test-slave-zw7p"
server_key: "e5f81d60c86b1fcd11ba5836284ff284"
server_key_iv: "59d27e2810d1fcbc80556478b91066f0"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:22.652340 16971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "c86cd9c448e74cf5a5b84c914e9c2664"
format_stamp: "Formatted at 2026-09-27 07:11:22 on dist-test-slave-zw7p"
server_key: "e5f81d60c86b1fcd11ba5836284ff284"
server_key_iv: "59d27e2810d1fcbc80556478b91066f0"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:22.654067 16971 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260927 07:11:22.672618 16986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:22.673017 16971 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260927 07:11:22.673142 16971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "c86cd9c448e74cf5a5b84c914e9c2664"
format_stamp: "Formatted at 2026-09-27 07:11:22 on dist-test-slave-zw7p"
server_key: "e5f81d60c86b1fcd11ba5836284ff284"
server_key_iv: "59d27e2810d1fcbc80556478b91066f0"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:22.673237 16971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:22.701221 16971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:22.703167 16971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:22.703305 16971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:22.707681 16971 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40767
I20260927 07:11:22.707763 17038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40767 every 8 connection(s)
I20260927 07:11:22.708091 16971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:11:22.708745 17039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:22.709524 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 16971
I20260927 07:11:22.709614 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:11:22.709810 4793 external_mini_cluster.cc:1428] Setting key cfd2374ae24135e73b90721c0265d8ae
I20260927 07:11:22.712131 17039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Bootstrap starting.
I20260927 07:11:22.712846 17039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:22.713146 17039 log.cc:824] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:22.713864 17039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: No bootstrap required, opened a new log
I20260927 07:11:22.716107 17039 raft_consensus.cc:348] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } }
I20260927 07:11:22.716249 17039 raft_consensus.cc:374] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:22.716284 17039 raft_consensus.cc:729] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86cd9c448e74cf5a5b84c914e9c2664, State: Initialized, Role: FOLLOWER
I20260927 07:11:22.716461 17039 consensus_queue.cc:260] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } }
I20260927 07:11:22.716540 17039 raft_consensus.cc:388] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:11:22.716571 17039 raft_consensus.cc:482] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:11:22.716632 17039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:22.717180 17039 raft_consensus.cc:504] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } }
I20260927 07:11:22.717291 17039 leader_election.cc:302] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c86cd9c448e74cf5a5b84c914e9c2664; no voters:
I20260927 07:11:22.717478 17039 leader_election.cc:288] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:11:22.717521 17044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 1 FOLLOWER]: Leader election won for term 1
Sep 27 07:11:22 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:22.717866 17039 sys_catalog.cc:564] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:22.718236 17044 raft_consensus.cc:686] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 1 LEADER]: Becoming Leader. State: Replica: c86cd9c448e74cf5a5b84c914e9c2664, State: Running, Role: LEADER
I20260927 07:11:22.718389 17044 consensus_queue.cc:237] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } }
I20260927 07:11:22.718811 17044 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } } }
I20260927 07:11:22.718938 17044 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:22.719105 17044 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c86cd9c448e74cf5a5b84c914e9c2664. Latest consensus state: current_term: 1 leader_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } } }
I20260927 07:11:22.719198 17044 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:22.719280 17039 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 07:11:22.719337 17058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:11:22.719990 17058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:11:22.722059 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:22.710343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.4.174.126:40767)
0927 07:11:22.710648 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:22.710659 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:22.710695 (+ 36us) server_negotiation.cc:407] Connection header received
0927 07:11:22.711288 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:22.711296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:22.711362 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:22.711524 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:22.712301 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:22.713566 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:22.714762 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:22.715400 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:22.715513 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:22.718191 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:22.718195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:22.718195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:22.718207 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:22.720192 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:22.720579 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:22.720582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:22.720583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:22.720675 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:22.720861 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:22.720862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:22.720863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:22.721154 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:22.721649 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:22.721728 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 07:11:22.721849 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":67,"threads_started":1}
I20260927 07:11:22.726395 17061 mvcc.cc:205] Tried to move back new op lower bound from 7333859666844856320 to 7333859666815225856. Current Snapshot: MvccSnapshot[applied={T|T < 7333859666844856320}]
I20260927 07:11:22.730708 17058 catalog_manager.cc:1347] Generated new cluster ID: 3846bb5d417942dd84b920974ea115b3
I20260927 07:11:22.730756 17058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:11:22.743527 17058 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:11:22.744346 17058 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:11:22.748921 17058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Generated new TSK 0
I20260927 07:11:22.749190 17058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:11:23.485670 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40767
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:23.593048 17084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:23.593263 17084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:23.593297 17084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:23.594516 17084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:23.594579 17084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:23.594616 17084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:11:23.596146 17084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.174.84:42671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:23.596372 17084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:23.596608 17084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:23.601435 17089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:23.601506 17092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:23.601697 17090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:23.601822 17084 server_base.cc:1034] running on GCE node
I20260927 07:11:23.602037 17084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:23.602250 17084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:23.603397 17084 hybrid_clock.cc:648] HybridClock initialized: now 1790493083603373 us; error 38 us; skew 500 ppm
Sep 27 07:11:23 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:23.605871 17084 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:11:23.606268 17084 webserver.cc:460] Webserver started at http://127.4.174.65:44245/ using document root <none> and password file <none>
I20260927 07:11:23.606482 17084 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:23.606528 17084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:23.606678 17084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:11:23 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:23.626065 17084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "0d218c78792045b29b3d8fe9de796bec"
format_stamp: "Formatted at 2026-09-27 07:11:23 on dist-test-slave-zw7p"
server_key: "94c2448f25e0e5874820eb210f0bc968"
server_key_iv: "443679911755ce38666f56a7d317dde5"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:23.626631 17084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "0d218c78792045b29b3d8fe9de796bec"
format_stamp: "Formatted at 2026-09-27 07:11:23 on dist-test-slave-zw7p"
server_key: "94c2448f25e0e5874820eb210f0bc968"
server_key_iv: "443679911755ce38666f56a7d317dde5"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:23.628017 17084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:11:23.644176 17099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:23.644454 17084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260927 07:11:23.644583 17084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "0d218c78792045b29b3d8fe9de796bec"
format_stamp: "Formatted at 2026-09-27 07:11:23 on dist-test-slave-zw7p"
server_key: "94c2448f25e0e5874820eb210f0bc968"
server_key_iv: "443679911755ce38666f56a7d317dde5"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:23.644723 17084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:23.658725 17084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:23.660926 17084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:23.661103 17084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:23.661360 17084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:23.661734 17084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:23.661801 17084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:23.661854 17084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:23.661895 17084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:23.668438 17084 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:42825
I20260927 07:11:23.668869 17084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:11:23.669461 17212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:42825 every 8 connection(s)
I20260927 07:11:23.677582 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 17084
I20260927 07:11:23.677691 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:11:23.677879 4793 external_mini_cluster.cc:1428] Setting key bee86ea50fcacfad620ac10b2521e342
Sep 27 07:11:23 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:23.681612 17215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:23.669618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40767 (local address 127.4.174.65:38591)
0927 07:11:23.670658 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:23.670706 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:23.671317 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:23.672797 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:23.672802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:23.673036 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:23.673316 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:23.673323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:23.674310 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:23.674311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:23.675531 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:23.675534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:23.676080 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:23.676081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:23.676211 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:23.676737 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:23.676746 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:23.678315 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:23.680181 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:23.680187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:23.680190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:23.680458 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:23.680644 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:23.680645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:23.680645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:23.680757 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:23.681003 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:23.681008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:23.681155 (+ 147us) client_negotiation.cc:769] Sending connection context
0927 07:11:23.681248 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 07:11:23.681383 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":76,"threads_started":1}
I20260927 07:11:23.682111 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:23.669642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:38591 (local address 127.4.174.126:40767)
0927 07:11:23.669925 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:23.669927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:23.670799 (+ 872us) server_negotiation.cc:407] Connection header received
0927 07:11:23.671416 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:23.671417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:23.671432 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:23.671521 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:23.673426 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:23.674210 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:23.675622 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:23.675954 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:23.676007 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:23.678410 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:23.678412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:23.678412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:23.678419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:23.680100 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:23.680531 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:23.680532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:23.680533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:23.680589 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:23.680813 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:23.680813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:23.680814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:23.680932 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:23.680981 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:23.681936 (+ 955us) server_negotiation.cc:299] Negotiation successful
0927 07:11:23.682015 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":71,"threads_started":1}
I20260927 07:11:23.682796 17213 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40767
I20260927 07:11:23.682880 17213 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:23.683049 17213 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:23.683650 17003 ts_manager.cc:194] Registered new tserver with Master: 0d218c78792045b29b3d8fe9de796bec (127.4.174.65:42825)
I20260927 07:11:23.684497 17003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:38591
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:11:23.727393 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40767
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:23.843538 17221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:23.843890 17221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:23.843974 17221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:23.845742 17221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:23.845849 17221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:23.845935 17221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:11:23.848168 17221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.174.84:42671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:23.848649 17221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:23.848996 17221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:23.855290 17228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:23.855398 17231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:23.855273 17229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:23.856433 17221 server_base.cc:1034] running on GCE node
I20260927 07:11:23.856734 17221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:23.856998 17221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:23.858151 17221 hybrid_clock.cc:648] HybridClock initialized: now 1790493083858125 us; error 40 us; skew 500 ppm
Sep 27 07:11:23 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:23.861377 17221 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:11:23.861927 17221 webserver.cc:460] Webserver started at http://127.4.174.66:38727/ using document root <none> and password file <none>
I20260927 07:11:23.862190 17221 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:23.862249 17221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:23.862424 17221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:11:23 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
Sep 27 07:11:23 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790493083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:23.883189 17221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "4e2fc67991b34a98bc496fc242b53d1d"
format_stamp: "Formatted at 2026-09-27 07:11:23 on dist-test-slave-zw7p"
server_key: "1cb06d6652bd71524a0359666e2f6be9"
server_key_iv: "3f006e756f84310c3b9bb693c766660b"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:23.883888 17221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "4e2fc67991b34a98bc496fc242b53d1d"
format_stamp: "Formatted at 2026-09-27 07:11:23 on dist-test-slave-zw7p"
server_key: "1cb06d6652bd71524a0359666e2f6be9"
server_key_iv: "3f006e756f84310c3b9bb693c766660b"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:23.890735 17221 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260927 07:11:23.911293 17238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:23.911810 17221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 07:11:23.911911 17221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "4e2fc67991b34a98bc496fc242b53d1d"
format_stamp: "Formatted at 2026-09-27 07:11:23 on dist-test-slave-zw7p"
server_key: "1cb06d6652bd71524a0359666e2f6be9"
server_key_iv: "3f006e756f84310c3b9bb693c766660b"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:23.912014 17221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:23.929607 17221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:23.932035 17221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:23.932226 17221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:23.932484 17221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:23.932860 17221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:23.932902 17221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:23.932931 17221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:23.932948 17221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:23.939630 17221 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:39991
I20260927 07:11:23.940127 17221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:11:23.940986 17351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:39991 every 8 connection(s)
I20260927 07:11:23.945812 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 17221
I20260927 07:11:23.945905 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:11:23.946115 4793 external_mini_cluster.cc:1428] Setting key 369a474c78975b786029734c440541c3
Sep 27 07:11:23 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:23.952826 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:23.941584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:39901 (local address 127.4.174.126:40767)
0927 07:11:23.941649 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:23.941651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:23.941866 (+ 215us) server_negotiation.cc:407] Connection header received
0927 07:11:23.942365 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:23.942366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:23.942384 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:23.942435 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:23.943033 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:23.943542 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:23.944498 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:23.944703 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:23.944756 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:23.949320 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:23.949322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:23.949322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:23.949331 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:23.951090 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:23.951654 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:23.951655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:23.951656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:23.951723 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:23.952042 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:23.952043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:23.952043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:23.952192 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:23.952262 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:23.952627 (+ 365us) server_negotiation.cc:299] Negotiation successful
0927 07:11:23.952729 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 07:11:23.953023 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:23.941555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40767 (local address 127.4.174.66:39901)
0927 07:11:23.941727 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:23.941771 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:23.942275 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:23.942507 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:23.942511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:23.942743 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:23.942940 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:23.942945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:23.943623 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:23.943624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:23.944415 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:23.944417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:23.944781 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:23.944783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:23.944896 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:23.945372 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:23.945380 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:23.949236 (+ 3856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:23.951185 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:23.951194 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:23.951197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:23.951547 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:23.951802 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:23.951803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:23.951803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:23.951962 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:23.952266 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:23.952272 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:23.952462 (+ 190us) client_negotiation.cc:769] Sending connection context
0927 07:11:23.952597 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 07:11:23.952774 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260927 07:11:23.953670 17352 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40767
I20260927 07:11:23.953794 17352 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:23.953999 17352 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:23.954444 17003 ts_manager.cc:194] Registered new tserver with Master: 4e2fc67991b34a98bc496fc242b53d1d (127.4.174.66:39991)
I20260927 07:11:23.954833 17003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:39901
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:11:23.975214 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40767
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:24.102176 17358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:24.102485 17358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:24.102527 17358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:24.104346 17358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:24.104408 17358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:24.104450 17358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:11:24.106671 17358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.174.84:42671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:24.107066 17358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:24.107383 17358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:24.113960 17364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:24.114375 17363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:24.114630 17366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:24.115810 17358 server_base.cc:1034] running on GCE node
I20260927 07:11:24.116017 17358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:24.116266 17358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:24.117417 17358 hybrid_clock.cc:648] HybridClock initialized: now 1790493084117397 us; error 34 us; skew 500 ppm
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:24.120465 17358 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:11:24.121059 17358 webserver.cc:460] Webserver started at http://127.4.174.67:41391/ using document root <none> and password file <none>
I20260927 07:11:24.121364 17358 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:24.121479 17358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:24.121668 17358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:24.135799 17358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c"
format_stamp: "Formatted at 2026-09-27 07:11:24 on dist-test-slave-zw7p"
server_key: "cfd6082f32da5d1b497dd7c3e4b36c0a"
server_key_iv: "5c4becaa581860703ec8ded1853ad58a"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:24.136274 17358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c"
format_stamp: "Formatted at 2026-09-27 07:11:24 on dist-test-slave-zw7p"
server_key: "cfd6082f32da5d1b497dd7c3e4b36c0a"
server_key_iv: "5c4becaa581860703ec8ded1853ad58a"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:24.139716 17358 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260927 07:11:24.158479 17373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.158742 17358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:11:24.158862 17358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c"
format_stamp: "Formatted at 2026-09-27 07:11:24 on dist-test-slave-zw7p"
server_key: "cfd6082f32da5d1b497dd7c3e4b36c0a"
server_key_iv: "5c4becaa581860703ec8ded1853ad58a"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:24.159010 17358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:24.220276 17358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:24.223821 17358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:24.223965 17358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:24.224212 17358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:24.224567 17358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:24.224604 17358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.224633 17358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:24.224654 17358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.231441 17358 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:40835
I20260927 07:11:24.231807 17358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:11:24.233355 17486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:40835 every 8 connection(s)
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:24.240947 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 17358
I20260927 07:11:24.241034 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:11:24.241222 4793 external_mini_cluster.cc:1428] Setting key e5fc220518f077316357fde9ce994620
I20260927 07:11:24.242668 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:24.233093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:59577 (local address 127.4.174.126:40767)
0927 07:11:24.233144 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:24.233146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:24.233641 (+ 495us) server_negotiation.cc:407] Connection header received
0927 07:11:24.234339 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:24.234340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:24.234354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:24.234407 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:24.235075 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.235568 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.236887 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.237193 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.237239 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:24.239667 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:24.239668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:24.239669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:24.239676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:24.241026 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:24.241616 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:24.241617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:24.241618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:24.241679 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:24.241957 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:24.241957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:24.241958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:24.242109 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:24.242150 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:24.242450 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 07:11:24.242549 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:11:24.242903 17489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:24.232784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40767 (local address 127.4.174.67:59577)
0927 07:11:24.233512 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:24.233556 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:24.234053 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:24.234580 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:24.234585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:24.234783 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:24.234984 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:24.234989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.235652 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.235653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:24.236779 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:24.236783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.237346 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.237347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:24.237503 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:24.238120 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:24.238131 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:24.239580 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:24.241189 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:24.241197 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:24.241199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:24.241537 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:24.241751 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:24.241751 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:24.241752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:24.241887 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:24.242194 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:24.242200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:24.242324 (+ 124us) client_negotiation.cc:769] Sending connection context
0927 07:11:24.242411 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 07:11:24.242582 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":82,"threads_started":1}
I20260927 07:11:24.243388 17487 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40767
I20260927 07:11:24.243484 17487 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:24.243628 17487 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:24.243948 17003 ts_manager.cc:194] Registered new tserver with Master: 9328d7ecdbbc475fa87cffe3a4a43c6c (127.4.174.67:40835)
I20260927 07:11:24.244407 17003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:59577
I20260927 07:11:24.252942 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:11:24.253571 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 17084
I20260927 07:11:24.258620 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 17221
I20260927 07:11:24.263432 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 17358
I20260927 07:11:24.269538 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 16971
I20260927 07:11:24.275578 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40767
--webserver_interface=127.4.174.126
--webserver_port=40781
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40767
--ranger_config_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42671/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.174.84:42671/kms with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:24.353680 17490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:24.353883 17490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:24.353907 17490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:24.355063 17490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:24.355104 17490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:24.355118 17490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:24.355129 17490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:24.355140 17490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:24.356638 17490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.174.84:42671/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40767
--ranger_config_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=40781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:24.356848 17490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:24.357059 17490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:24.361568 17498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:24.361626 17495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:24.361626 17496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:24.361793 17490 server_base.cc:1034] running on GCE node
I20260927 07:11:24.362053 17490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:24.362321 17490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:24.363456 17490 hybrid_clock.cc:648] HybridClock initialized: now 1790493084363429 us; error 48 us; skew 500 ppm
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:24.365706 17490 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:11:24.366164 17490 webserver.cc:460] Webserver started at http://127.4.174.126:40781/ using document root <none> and password file <none>
I20260927 07:11:24.366367 17490 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:24.366420 17490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:24.429088 17490 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:24.429975 17506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.430168 17490 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:24.430269 17490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "c86cd9c448e74cf5a5b84c914e9c2664"
format_stamp: "Formatted at 2026-09-27 07:11:22 on dist-test-slave-zw7p"
server_key: "e5f81d60c86b1fcd11ba5836284ff284"
server_key_iv: "59d27e2810d1fcbc80556478b91066f0"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:24.430800 17490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:24.449539 17490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:24.451903 17490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:24.452065 17490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:24.457031 17490 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40767
I20260927 07:11:24.457059 17558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40767 every 8 connection(s)
I20260927 07:11:24.457537 17490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:11:24.458866 17559 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:11:24.459506 17559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Bootstrap starting.
I20260927 07:11:24.461269 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 17490
I20260927 07:11:24.461570 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:42825
--local_ip_for_outbound_sockets=127.4.174.65
--tserver_master_addrs=127.4.174.126:40767
--webserver_port=44245
--webserver_interface=127.4.174.65
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.174.84:42671/kms with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:11:24.467519 17559 log.cc:824] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:24.469623 17559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:11:24.469926 17559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Bootstrap complete.
I20260927 07:11:24.473187 17559 raft_consensus.cc:348] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } }
I20260927 07:11:24.473505 17559 raft_consensus.cc:729] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86cd9c448e74cf5a5b84c914e9c2664, State: Initialized, Role: FOLLOWER
I20260927 07:11:24.473713 17559 consensus_queue.cc:260] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } }
I20260927 07:11:24.473795 17559 raft_consensus.cc:388] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:11:24.473845 17559 raft_consensus.cc:482] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:11:24.473891 17559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:11:24.475250 17559 raft_consensus.cc:504] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } }
I20260927 07:11:24.475358 17559 leader_election.cc:302] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c86cd9c448e74cf5a5b84c914e9c2664; no voters:
I20260927 07:11:24.475525 17559 leader_election.cc:288] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 07:11:24.475582 17563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 07:11:24.475811 17563 raft_consensus.cc:686] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [term 2 LEADER]: Becoming Leader. State: Replica: c86cd9c448e74cf5a5b84c914e9c2664, State: Running, Role: LEADER
I20260927 07:11:24.475879 17559 sys_catalog.cc:564] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:24.475992 17563 consensus_queue.cc:237] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } }
I20260927 07:11:24.476311 17563 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } } }
I20260927 07:11:24.476469 17563 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:24.476637 17563 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c86cd9c448e74cf5a5b84c914e9c2664. Latest consensus state: current_term: 2 leader_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86cd9c448e74cf5a5b84c914e9c2664" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40767 } } }
I20260927 07:11:24.476727 17563 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664 [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:24.476851 17572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:11:24.477325 17572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:11:24.477509 17572 catalog_manager.cc:1259] Loaded cluster ID: 3846bb5d417942dd84b920974ea115b3
I20260927 07:11:24.477576 17572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:11:24.477766 17559 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 07:11:24.481330 17572 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:11:24.481954 17572 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Loaded TSK: 0
I20260927 07:11:24.482270 17572 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 07:11:24.571264 17561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:24.571456 17561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:24.571488 17561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:24.572723 17561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:24.572768 17561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:24.572803 17561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:11:24.574290 17561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.174.84:42671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:42825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=44245
--tserver_master_addrs=127.4.174.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:24.574519 17561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:24.574772 17561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:24.579715 17599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:24.580708 17596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:24.583606 17561 server_base.cc:1034] running on GCE node
W20260927 07:11:24.584507 17597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:24.584771 17561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:24.585038 17561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:24.586237 17561 hybrid_clock.cc:648] HybridClock initialized: now 1790493084586213 us; error 33 us; skew 500 ppm
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:24.589293 17561 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:11:24.589798 17561 webserver.cc:460] Webserver started at http://127.4.174.65:44245/ using document root <none> and password file <none>
I20260927 07:11:24.590041 17561 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:24.590113 17561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:24.613359 17561 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260927 07:11:24.614362 17607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.618199 17561 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260927 07:11:24.618307 17561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "0d218c78792045b29b3d8fe9de796bec"
format_stamp: "Formatted at 2026-09-27 07:11:23 on dist-test-slave-zw7p"
server_key: "94c2448f25e0e5874820eb210f0bc968"
server_key_iv: "443679911755ce38666f56a7d317dde5"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:24.618834 17561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:24.666917 17561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:24.669219 17561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:24.669401 17561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:24.669680 17561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:24.670086 17561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:24.670138 17561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.670171 17561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:24.670187 17561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.677928 17561 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:42825
I20260927 07:11:24.678550 17561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:11:24.678743 17720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:42825 every 8 connection(s)
I20260927 07:11:24.681675 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 17561
I20260927 07:11:24.681967 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:39991
--local_ip_for_outbound_sockets=127.4.174.66
--tserver_master_addrs=127.4.174.126:40767
--webserver_port=38727
--webserver_interface=127.4.174.66
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.174.84:42671/kms with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:24.691356 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:24.679834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:36597 (local address 127.4.174.126:40767)
0927 07:11:24.680062 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:24.680071 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:24.680117 (+ 46us) server_negotiation.cc:407] Connection header received
0927 07:11:24.680902 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:24.680911 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:24.680978 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:24.681126 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:24.681881 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.683051 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.684334 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.684770 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.684868 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:24.687325 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:24.687330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:24.687331 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:24.687343 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:24.689256 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:24.689756 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:24.689759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:24.689760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:24.689874 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:24.690178 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:24.690179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:24.690179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:24.690483 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:24.690544 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:24.690967 (+ 423us) server_negotiation.cc:299] Negotiation successful
0927 07:11:24.691125 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260927 07:11:24.692247 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:24.679754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40767 (local address 127.4.174.65:36597)
0927 07:11:24.679944 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:24.679995 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:24.680625 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:24.681195 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:24.681200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:24.681440 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:24.681728 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:24.681735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.683167 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.683169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:24.684242 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:24.684245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.684967 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.684969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:24.685076 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:24.685600 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:24.685610 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:24.687193 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:24.689352 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:24.689360 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:24.689362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:24.689655 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:24.689953 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:24.689954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:24.689954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:24.690097 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:24.690618 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:24.690624 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:24.690778 (+ 154us) client_negotiation.cc:769] Sending connection context
0927 07:11:24.691838 (+ 1060us) client_negotiation.cc:240] Negotiation successful
0927 07:11:24.692021 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260927 07:11:24.693253 17721 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40767
I20260927 07:11:24.693359 17721 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:24.693570 17721 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
W20260927 07:11:24.694494 17721 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40767 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 07:11:24.694592 17721 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:24.694666 17721 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:24.695015 17721 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:24.695094 17721 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
W20260927 07:11:24.695405 17721 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:11:24.804525 17725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:24.804869 17725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:24.804949 17725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:24.806722 17725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:24.806816 17725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:24.806891 17725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:11:24.809181 17725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.174.84:42671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:39991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=38727
--tserver_master_addrs=127.4.174.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:24.809579 17725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:24.809901 17725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:24.815178 17730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:24.815344 17733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:24.815346 17725 server_base.cc:1034] running on GCE node
W20260927 07:11:24.815680 17731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:24.815936 17725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:24.816214 17725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:24.817443 17725 hybrid_clock.cc:648] HybridClock initialized: now 1790493084817416 us; error 38 us; skew 500 ppm
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:24.820530 17725 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:11:24.821076 17725 webserver.cc:460] Webserver started at http://127.4.174.66:38727/ using document root <none> and password file <none>
I20260927 07:11:24.821336 17725 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:24.821413 17725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:24.846805 17725 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260927 07:11:24.847976 17741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.848189 17725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 07:11:24.848277 17725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "4e2fc67991b34a98bc496fc242b53d1d"
format_stamp: "Formatted at 2026-09-27 07:11:23 on dist-test-slave-zw7p"
server_key: "1cb06d6652bd71524a0359666e2f6be9"
server_key_iv: "3f006e756f84310c3b9bb693c766660b"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:24.848644 17725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:24.856285 17725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:24.858121 17725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:24.858299 17725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:24.858506 17725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:24.858812 17725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:24.858846 17725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.858875 17725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:24.858927 17725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:24.865177 17725 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:39991
I20260927 07:11:24.865274 17854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:39991 every 8 connection(s)
I20260927 07:11:24.865592 17725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:11:24.868489 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 17725
I20260927 07:11:24.868760 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.174.84:42671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:40835
--local_ip_for_outbound_sockets=127.4.174.67
--tserver_master_addrs=127.4.174.126:40767
--webserver_port=41391
--webserver_interface=127.4.174.67
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.174.84:42671/kms with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:11:24 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493084, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:24.875311 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:24.866362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:45895 (local address 127.4.174.126:40767)
0927 07:11:24.866408 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:24.866410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:24.866645 (+ 235us) server_negotiation.cc:407] Connection header received
0927 07:11:24.867144 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:24.867146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:24.867162 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:24.867211 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:24.867742 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.868233 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.869582 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.869853 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.869898 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:24.872204 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:24.872205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:24.872206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:24.872213 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:24.873754 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:24.874240 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:24.874241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:24.874242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:24.874304 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:24.874527 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:24.874528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:24.874529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:24.874681 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:24.874741 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:24.875123 (+ 382us) server_negotiation.cc:299] Negotiation successful
0927 07:11:24.875209 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:11:24.875800 17857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:24.866364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40767 (local address 127.4.174.66:45895)
0927 07:11:24.866514 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:24.866572 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:24.867048 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:24.867281 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:24.867285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:24.867461 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:24.867652 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:24.867656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.868322 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.868323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:24.869483 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:24.869486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:24.870033 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:24.870034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:24.870170 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:24.870717 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:24.870726 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:24.872094 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:24.873841 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:24.873850 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:24.873852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:24.874148 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:24.874364 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:24.874365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:24.874365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:24.874462 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:24.874768 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:24.874774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:24.874964 (+ 190us) client_negotiation.cc:769] Sending connection context
0927 07:11:24.875425 (+ 461us) client_negotiation.cc:240] Negotiation successful
0927 07:11:24.875582 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260927 07:11:24.876329 17855 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40767
I20260927 07:11:24.876423 17855 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:24.876587 17855 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
W20260927 07:11:24.877195 17855 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40767 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 07:11:24.877261 17855 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:24.877319 17855 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:24.877658 17855 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:24.877733 17855 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
W20260927 07:11:24.878005 17855 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:11:24.980288 17858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:24.980639 17858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:24.980715 17858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:24.982456 17858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:24.982587 17858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:24.982669 17858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:11:24.984808 17858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.174.84:42671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:40835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=41391
--tserver_master_addrs=127.4.174.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:24.985246 17858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:24.985553 17858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:24.994958 17864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:24.994952 17863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:24.996294 17858 server_base.cc:1034] running on GCE node
W20260927 07:11:24.998608 17866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:24.998888 17858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:24.999187 17858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:25.000344 17858 hybrid_clock.cc:648] HybridClock initialized: now 1790493085000323 us; error 37 us; skew 500 ppm
Sep 27 07:11:25 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493085, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:25.003556 17858 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:11:25.004037 17858 webserver.cc:460] Webserver started at http://127.4.174.67:41391/ using document root <none> and password file <none>
I20260927 07:11:25.004271 17858 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:25.004333 17858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 07:11:25 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493085, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:25.024922 17858 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:25.025990 17875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:25.026206 17858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 07:11:25.026294 17858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c"
format_stamp: "Formatted at 2026-09-27 07:11:24 on dist-test-slave-zw7p"
server_key: "cfd6082f32da5d1b497dd7c3e4b36c0a"
server_key_iv: "5c4becaa581860703ec8ded1853ad58a"
server_key_version: "kuduclusterkey@0"
I20260927 07:11:25.026827 17858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:25.044997 17858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:25.047333 17858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:25.047552 17858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:25.047850 17858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:25.048255 17858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:25.048341 17858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:25.048424 17858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:25.048478 17858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:25.054427 17858 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:40835
I20260927 07:11:25.054950 17858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:11:25.057571 17988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:40835 every 8 connection(s)
I20260927 07:11:25.064289 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 17858
Sep 27 07:11:25 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493085, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:25.069442 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:25.058491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:39081 (local address 127.4.174.126:40767)
0927 07:11:25.058538 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:25.058540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:25.059376 (+ 836us) server_negotiation.cc:407] Connection header received
0927 07:11:25.060079 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:25.060081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:25.060095 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:25.060123 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:25.060907 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:25.061674 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:25.063202 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:25.063500 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:25.063539 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:25.066351 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:25.066352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:25.066353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:25.066360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:25.067975 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:25.068429 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:25.068430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:25.068431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:25.068485 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:25.068730 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:25.068731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:25.068731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:25.068856 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:25.068894 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:25.069259 (+ 365us) server_negotiation.cc:299] Negotiation successful
0927 07:11:25.069344 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:11:25.070197 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:25.058182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40767 (local address 127.4.174.67:39081)
0927 07:11:25.059230 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:25.059279 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:25.059953 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:25.060303 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:25.060307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:25.060530 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:25.060800 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:25.060806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:25.061785 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:25.061786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:25.063103 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:25.063107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:25.063642 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:25.063643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:25.063809 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:25.064386 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:25.064396 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:25.066249 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:25.068066 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:25.068072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:25.068075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:25.068349 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:25.068547 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:25.068548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:25.068548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:25.068669 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:25.068947 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:25.068952 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:25.069117 (+ 165us) client_negotiation.cc:769] Sending connection context
0927 07:11:25.069793 (+ 676us) client_negotiation.cc:240] Negotiation successful
0927 07:11:25.069967 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"thread_start_us":78,"threads_started":1}
I20260927 07:11:25.070829 17989 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40767
I20260927 07:11:25.070933 17989 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:25.071116 17989 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
W20260927 07:11:25.071835 17989 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40767 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 07:11:25.071899 17989 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:25.071964 17989 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:25.072268 17989 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:25.072331 17989 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
W20260927 07:11:25.072608 17989 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:11:25.080199 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:25.064684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48918 (local address 127.4.174.126:40767)
0927 07:11:25.069513 (+ 4829us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:25.069514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:25.072928 (+ 3414us) server_negotiation.cc:407] Connection header received
0927 07:11:25.073049 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:25.073050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:25.073058 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:25.073084 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:25.073819 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:25.074550 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:25.075696 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:25.075987 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:25.076023 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:25.077454 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:25.077455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:25.077456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:25.077462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:25.078964 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:25.079332 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:25.079333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:25.079334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:25.079386 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:25.079565 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:25.079566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:25.079566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:25.079678 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:25.079824 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:25.080036 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 07:11:25.080107 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4815,"thread_start_us":71,"threads_started":1}
Sep 27 07:11:25 dist-test-slave-zw7p krb5kdc[16527](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790493085, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:25.695577 17721 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:25.695739 17721 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:25.696107 17523 ts_manager.cc:194] Registered new tserver with Master: 0d218c78792045b29b3d8fe9de796bec (127.4.174.65:42825)
I20260927 07:11:25.697119 17523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:36597
I20260927 07:11:25.878161 17855 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:25.878288 17855 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:25.878618 17523 ts_manager.cc:194] Registered new tserver with Master: 4e2fc67991b34a98bc496fc242b53d1d (127.4.174.66:39991)
I20260927 07:11:25.879129 17523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:45895
Sep 27 07:11:26 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493085, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:26.072836 17989 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:26.072999 17989 heartbeater.cc:511] Master 127.4.174.126:40767 requested a full tablet report, sending...
I20260927 07:11:26.073287 17523 ts_manager.cc:194] Registered new tserver with Master: 9328d7ecdbbc475fa87cffe3a4a43c6c (127.4.174.67:40835)
I20260927 07:11:26.073791 17523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:39081
I20260927 07:11:26.082543 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:11:26.096745 18011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.083831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48928 (local address 127.4.174.126:40767)
0927 07:11:26.083964 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:26.083967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:26.083974 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:11:26.083994 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:26.083994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:26.084003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:26.084058 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:26.084586 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.085203 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.086079 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.086317 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.086372 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.087568 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:26.087569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:26.087569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:26.087577 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:26.095143 (+ 7566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:26.095845 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:26.095846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:26.095846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:26.095916 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:26.096158 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:26.096159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:26.096159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:26.096315 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:26.096422 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:26.096538 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 07:11:26.096631 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
I20260927 07:11:26.103371 17523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:11:26.105789 17523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:11:26.121287 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.112552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42825 (local address 127.0.0.1:37180)
0927 07:11:26.113848 (+ 1296us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:26.113905 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:26.113965 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:26.114394 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:26.114397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:26.114398 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:26.114476 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.114478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.116125 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.116126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.117433 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.117436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.120982 (+ 3546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.120983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.121072 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.121086 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 07:11:26.121148 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 07:11:26.121200 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"thread_start_us":77,"threads_started":1}
I20260927 07:11:26.122725 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.112750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39991 (local address 127.0.0.1:40216)
0927 07:11:26.114056 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:26.114063 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:26.114093 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:26.114817 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:26.114818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:26.114820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:26.114873 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.114874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.118760 (+ 3886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.118761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.119953 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.119956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.122457 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.122458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.122544 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.122547 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:11:26.122613 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 07:11:26.122652 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1295,"thread_start_us":62,"threads_started":1}
I20260927 07:11:26.123234 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.113001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.4.174.65:42825)
0927 07:11:26.114180 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:26.114184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:26.114193 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:11:26.114226 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:26.114228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:26.114242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:26.114303 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:26.114944 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.116036 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.120043 (+ 4007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.120822 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.120888 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.120949 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:26.122925 (+ 1976us) server_negotiation.cc:299] Negotiation successful
0927 07:11:26.122970 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1149,"thread_start_us":52,"threads_started":1}
I20260927 07:11:26.123504 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.113126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.4.174.66:39991)
0927 07:11:26.114612 (+ 1486us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:26.114616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:26.114625 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:11:26.114658 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:26.114660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:26.114674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:26.114740 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:26.117539 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.118676 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.121492 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.122293 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.122365 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.122422 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:26.123030 (+ 608us) server_negotiation.cc:299] Negotiation successful
0927 07:11:26.123076 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1453,"thread_start_us":39,"threads_started":1}
I20260927 07:11:26.123919 17639 tablet_service.cc:1467] Processing CreateTablet for tablet fae1686c9d2c47f79091f5a10cfc3acd (DEFAULT_TABLE table=test-table [id=55b969481faa43aaa85e358efe81bfc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:26.124087 17789 tablet_service.cc:1467] Processing CreateTablet for tablet fae1686c9d2c47f79091f5a10cfc3acd (DEFAULT_TABLE table=test-table [id=55b969481faa43aaa85e358efe81bfc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:26.124323 17639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae1686c9d2c47f79091f5a10cfc3acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:26.124349 17789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae1686c9d2c47f79091f5a10cfc3acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:26.126451 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.113289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:40835 (local address 127.0.0.1:49514)
0927 07:11:26.121233 (+ 7944us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:26.121244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:26.121298 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:26.121844 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:26.121845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:26.121847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:26.121887 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.121889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.123471 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.123471 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.125214 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.125217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.126217 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.126218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.126279 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.126283 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:26.126330 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 07:11:26.126371 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7925,"thread_start_us":56,"threads_started":1}
I20260927 07:11:26.126827 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.113515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49514 (local address 127.4.174.67:40835)
0927 07:11:26.113784 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:26.113790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:26.121614 (+ 7824us) server_negotiation.cc:407] Connection header received
0927 07:11:26.121662 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:26.121665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:26.121681 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:26.121757 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:26.121970 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.123223 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.125311 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.126052 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.126122 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.126182 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:26.126702 (+ 520us) server_negotiation.cc:299] Negotiation successful
0927 07:11:26.126747 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":48,"threads_started":1}
I20260927 07:11:26.127787 17923 tablet_service.cc:1467] Processing CreateTablet for tablet fae1686c9d2c47f79091f5a10cfc3acd (DEFAULT_TABLE table=test-table [id=55b969481faa43aaa85e358efe81bfc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:26.128139 17923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae1686c9d2c47f79091f5a10cfc3acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:26.128258 18022 tablet_bootstrap.cc:492] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d: Bootstrap starting.
I20260927 07:11:26.128888 18022 tablet_bootstrap.cc:654] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:26.129189 18022 log.cc:824] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:26.129923 18022 tablet_bootstrap.cc:492] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d: No bootstrap required, opened a new log
I20260927 07:11:26.129993 18022 ts_tablet_manager.cc:1397] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:11:26.132074 18022 raft_consensus.cc:348] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.132242 18022 raft_consensus.cc:374] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:26.132275 18022 raft_consensus.cc:729] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2fc67991b34a98bc496fc242b53d1d, State: Initialized, Role: FOLLOWER
I20260927 07:11:26.132383 18022 consensus_queue.cc:260] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.132629 18022 ts_tablet_manager.cc:1428] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:11:26.132794 17855 heartbeater.cc:503] Master 127.4.174.126:40767 was elected leader, sending a full tablet report...
I20260927 07:11:26.134157 18023 tablet_bootstrap.cc:492] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec: Bootstrap starting.
I20260927 07:11:26.134881 18023 tablet_bootstrap.cc:654] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:26.135217 18023 log.cc:824] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:26.135978 18023 tablet_bootstrap.cc:492] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec: No bootstrap required, opened a new log
I20260927 07:11:26.136046 18023 ts_tablet_manager.cc:1397] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:11:26.138656 18023 raft_consensus.cc:348] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.138828 18023 raft_consensus.cc:374] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:26.138863 18023 raft_consensus.cc:729] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d218c78792045b29b3d8fe9de796bec, State: Initialized, Role: FOLLOWER
I20260927 07:11:26.139050 18023 consensus_queue.cc:260] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.139063 18025 tablet_bootstrap.cc:492] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Bootstrap starting.
I20260927 07:11:26.139300 18023 ts_tablet_manager.cc:1428] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:11:26.139411 17721 heartbeater.cc:503] Master 127.4.174.126:40767 was elected leader, sending a full tablet report...
I20260927 07:11:26.140542 18025 tablet_bootstrap.cc:654] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:26.140920 18025 log.cc:824] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:26.141734 18025 tablet_bootstrap.cc:492] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: No bootstrap required, opened a new log
I20260927 07:11:26.141805 18025 ts_tablet_manager.cc:1397] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:11:26.144556 18025 raft_consensus.cc:348] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.144806 18025 raft_consensus.cc:374] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:26.144891 18025 raft_consensus.cc:729] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9328d7ecdbbc475fa87cffe3a4a43c6c, State: Initialized, Role: FOLLOWER
I20260927 07:11:26.145093 18025 consensus_queue.cc:260] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.145421 18025 ts_tablet_manager.cc:1428] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260927 07:11:26.145581 17989 heartbeater.cc:503] Master 127.4.174.126:40767 was elected leader, sending a full tablet report...
W20260927 07:11:26.180133 17722 tablet.cc:2369] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:11:26.292686 18030 raft_consensus.cc:482] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:26.292852 18030 raft_consensus.cc:504] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.293246 18030 leader_election.cc:288] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d218c78792045b29b3d8fe9de796bec (127.4.174.65:42825), 4e2fc67991b34a98bc496fc242b53d1d (127.4.174.66:39991)
I20260927 07:11:26.297221 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.293551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:39921 (local address 127.4.174.65:42825)
0927 07:11:26.293596 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:26.293598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:26.293861 (+ 263us) server_negotiation.cc:407] Connection header received
0927 07:11:26.293950 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:26.293951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:26.293960 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:26.293985 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:26.294190 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.294950 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.296354 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.296865 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.296907 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.296916 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:26.297116 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 07:11:26.297153 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:11:26.297380 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.293926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:32939 (local address 127.4.174.66:39991)
0927 07:11:26.293972 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:26.293974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:26.294075 (+ 101us) server_negotiation.cc:407] Connection header received
0927 07:11:26.294171 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:26.294172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:26.294181 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:26.294206 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:26.294420 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.295109 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.296459 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.296990 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.297039 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.297048 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:26.297263 (+ 215us) server_negotiation.cc:299] Negotiation successful
0927 07:11:26.297309 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:11:26.297474 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.293445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42825 (local address 127.4.174.67:39921)
0927 07:11:26.293793 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:26.293811 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:26.293886 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:26.294048 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:26.294048 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:26.294050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:26.294126 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.294128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.295044 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.295044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.296260 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.296263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.296985 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.296986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.297046 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.297050 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:26.297330 (+ 280us) client_negotiation.cc:240] Negotiation successful
0927 07:11:26.297371 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":70,"threads_started":1}
I20260927 07:11:26.297807 17671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fae1686c9d2c47f79091f5a10cfc3acd" candidate_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d218c78792045b29b3d8fe9de796bec" is_pre_election: true
I20260927 07:11:26.298002 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.293627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39991 (local address 127.4.174.67:32939)
0927 07:11:26.294006 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:26.294015 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:26.294105 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:26.294265 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:26.294265 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:26.294267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:26.294346 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.294348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.295241 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.295242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.296354 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:26.296357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.297105 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.297106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:26.297181 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.297185 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:26.297862 (+ 677us) client_negotiation.cc:240] Negotiation successful
0927 07:11:26.297917 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":73,"threads_started":1}
I20260927 07:11:26.298004 17671 raft_consensus.cc:2454] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9328d7ecdbbc475fa87cffe3a4a43c6c in term 0.
I20260927 07:11:26.298326 17876 leader_election.cc:302] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d218c78792045b29b3d8fe9de796bec, 9328d7ecdbbc475fa87cffe3a4a43c6c; no voters:
I20260927 07:11:26.298328 17809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fae1686c9d2c47f79091f5a10cfc3acd" candidate_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2fc67991b34a98bc496fc242b53d1d" is_pre_election: true
I20260927 07:11:26.298451 17809 raft_consensus.cc:2454] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9328d7ecdbbc475fa87cffe3a4a43c6c in term 0.
I20260927 07:11:26.298475 18030 raft_consensus.cc:2781] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:11:26.298533 18030 raft_consensus.cc:482] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:11:26.298586 18030 raft_consensus.cc:3037] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:26.299419 18030 raft_consensus.cc:504] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.299559 18030 leader_election.cc:288] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [CANDIDATE]: Term 1 election: Requested vote from peers 0d218c78792045b29b3d8fe9de796bec (127.4.174.65:42825), 4e2fc67991b34a98bc496fc242b53d1d (127.4.174.66:39991)
I20260927 07:11:26.299862 17809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fae1686c9d2c47f79091f5a10cfc3acd" candidate_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2fc67991b34a98bc496fc242b53d1d"
I20260927 07:11:26.299863 17671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fae1686c9d2c47f79091f5a10cfc3acd" candidate_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d218c78792045b29b3d8fe9de796bec"
I20260927 07:11:26.299932 17809 raft_consensus.cc:3037] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:26.299955 17671 raft_consensus.cc:3037] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:26.300715 17809 raft_consensus.cc:2454] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9328d7ecdbbc475fa87cffe3a4a43c6c in term 1.
I20260927 07:11:26.300803 17671 raft_consensus.cc:2454] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9328d7ecdbbc475fa87cffe3a4a43c6c in term 1.
I20260927 07:11:26.300863 17876 leader_election.cc:302] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e2fc67991b34a98bc496fc242b53d1d, 9328d7ecdbbc475fa87cffe3a4a43c6c; no voters:
I20260927 07:11:26.300957 18030 raft_consensus.cc:2781] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:11:26.301113 18030 raft_consensus.cc:686] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 1 LEADER]: Becoming Leader. State: Replica: 9328d7ecdbbc475fa87cffe3a4a43c6c, State: Running, Role: LEADER
I20260927 07:11:26.301213 18030 consensus_queue.cc:237] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } }
I20260927 07:11:26.302081 17522 catalog_manager.cc:5495] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9328d7ecdbbc475fa87cffe3a4a43c6c (127.4.174.67). New cstate: current_term: 1 leader_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9328d7ecdbbc475fa87cffe3a4a43c6c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 } health_report { overall_health: UNKNOWN } } }
W20260927 07:11:26.309402 17990 tablet.cc:2369] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 07:11:26 dist-test-slave-zw7p krb5kdc[16527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493085, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for HTTP/127.4.174.84@KRBTEST.COM
I20260927 07:11:26.344304 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.340831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49528 (local address 127.4.174.67:40835)
0927 07:11:26.340870 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:26.340871 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:26.340877 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:11:26.340927 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:26.340928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:26.340949 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:26.341012 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:11:26.341201 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.341994 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.343200 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:26.343504 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:26.343561 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:26.343758 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:11:26.343914 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:11:26.343991 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:26.344202 (+ 211us) server_negotiation.cc:299] Negotiation successful
0927 07:11:26.344222 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:11:26.351615 17809 raft_consensus.cc:1261] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 1 FOLLOWER]: Refusing update from remote peer 9328d7ecdbbc475fa87cffe3a4a43c6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:26.352029 18030 consensus_queue.cc:1035] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e2fc67991b34a98bc496fc242b53d1d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:26.352649 17671 raft_consensus.cc:1261] T fae1686c9d2c47f79091f5a10cfc3acd P 0d218c78792045b29b3d8fe9de796bec [term 1 FOLLOWER]: Refusing update from remote peer 9328d7ecdbbc475fa87cffe3a4a43c6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:26.352916 18030 consensus_queue.cc:1035] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d218c78792045b29b3d8fe9de796bec" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:26.355470 18037 mvcc.cc:205] Tried to move back new op lower bound from 7333859681693466624 to 7333859681490292736. Current Snapshot: MvccSnapshot[applied={T|T < 7333859681693466624}]
I20260927 07:11:26.357523 17523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:11:26.357640 17523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:11:26.358314 17523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c86cd9c448e74cf5a5b84c914e9c2664: Sending DeleteTablet for 3 replicas of tablet fae1686c9d2c47f79091f5a10cfc3acd
I20260927 07:11:26.358711 17789 tablet_service.cc:1514] Processing DeleteTablet for tablet fae1686c9d2c47f79091f5a10cfc3acd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:26 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:40216
I20260927 07:11:26.358978 17639 tablet_service.cc:1514] Processing DeleteTablet for tablet fae1686c9d2c47f79091f5a10cfc3acd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:26 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:37180
I20260927 07:11:26.359297 17923 tablet_service.cc:1514] Processing DeleteTablet for tablet fae1686c9d2c47f79091f5a10cfc3acd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:26 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:49514
I20260927 07:11:26.359789 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 17561
I20260927 07:11:26.363554 18047 tablet_replica.cc:330] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: stopping tablet replica
I20260927 07:11:26.363656 18047 raft_consensus.cc:2229] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:11:26.363787 18047 raft_consensus.cc:2258] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:26.364120 18039 mvcc.cc:205] Tried to move back new op lower bound from 7333859681693466624 to 7333859681490292736. Current Snapshot: MvccSnapshot[applied={T|T < 7333859681693466624}]
I20260927 07:11:26.364427 18045 tablet_replica.cc:330] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d: stopping tablet replica
I20260927 07:11:26.364529 18045 raft_consensus.cc:2229] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:11:26.364596 18045 raft_consensus.cc:2258] T fae1686c9d2c47f79091f5a10cfc3acd P 4e2fc67991b34a98bc496fc242b53d1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:26.364588 18047 ts_tablet_manager.cc:1905] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:11:26.369938 18047 ts_tablet_manager.cc:1918] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:11:26.370030 18047 log.cc:1196] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/fae1686c9d2c47f79091f5a10cfc3acd
I20260927 07:11:26.370309 18047 ts_tablet_manager.cc:1939] T fae1686c9d2c47f79091f5a10cfc3acd P 9328d7ecdbbc475fa87cffe3a4a43c6c: Deleting consensus metadata
I20260927 07:11:26.370735 17508 catalog_manager.cc:4841] TS 9328d7ecdbbc475fa87cffe3a4a43c6c (127.4.174.67:40835): tablet fae1686c9d2c47f79091f5a10cfc3acd (table test-table [id=55b969481faa43aaa85e358efe81bfc7]) successfully deleted
W20260927 07:11:26.371513 17507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:42825 (error 108)
I20260927 07:11:26.372069 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 17725
I20260927 07:11:26.378866 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 17858
I20260927 07:11:26.382328 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.382114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:42825 (local address 127.0.0.1:37190)
0927 07:11:26.382236 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:26.382284 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260927 07:11:26.382442 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:26.382191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39991 (local address 127.0.0.1:40228)
0927 07:11:26.382404 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:26.382427 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.66:39991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
W20260927 07:11:26.382721 17507 catalog_manager.cc:4584] TS 0d218c78792045b29b3d8fe9de796bec (127.4.174.65:42825): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fae1686c9d2c47f79091f5a10cfc3acd: Network error: Client connection negotiation failed: client connection to 127.4.174.65:42825: connect: Connection refused (error 111)
I20260927 07:11:26.387406 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 17490
I20260927 07:11:26.393502 4793 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 07:11:26.737735 4793 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 07:11:26.737820 4793 mini_ranger.cc:66] Stopping Ranger...
I20260927 07:11:27.071730 4793 mini_ranger.cc:68] Stopped Ranger
2026-09-27 07:11:27.071 UTC [16557] LOG: received smart shutdown request
2026-09-27 07:11:27.075 UTC [16557] LOG: background worker "logical replication launcher" (PID 16566) exited with exit code 1
2026-09-27 07:11:27.075 UTC [16561] LOG: shutting down
2026-09-27 07:11:27.238 UTC [16557] LOG: database system is shut down
2026-09-27T07:11:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46524 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:11:27 dist-test-slave-zw7p krb5kdc[18058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:11:27 dist-test-slave-zw7p krb5kdc[18058](info): set up 2 sockets
Sep 27 07:11:27 dist-test-slave-zw7p krb5kdc[18058](info): commencing operation
krb5kdc: starting...
W20260927 07:11:28.274183 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:11:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:11:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:11:28.387682 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:38811
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:38811,127.4.174.125:46781,127.4.174.124:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:28.465902 18071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:28.466100 18071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:28.466130 18071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:28.467308 18071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:28.467350 18071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:28.467368 18071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:28.467386 18071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:28.467399 18071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:28.468873 18071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:38811,127.4.174.125:46781,127.4.174.124:37897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:38811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:28.469105 18071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:28.469303 18071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:28.473760 18076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:28.473739 18077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:28.473734 18079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:28.473830 18071 server_base.cc:1034] running on GCE node
I20260927 07:11:28.474174 18071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:28.474570 18071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:28.475759 18071 hybrid_clock.cc:648] HybridClock initialized: now 1790493088475743 us; error 31 us; skew 500 ppm
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:28.484079 18071 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:11:28.484526 18071 webserver.cc:460] Webserver started at http://127.4.174.126:40287/ using document root <none> and password file <none>
I20260927 07:11:28.484727 18071 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:28.484776 18071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:28.484901 18071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:28.485925 18071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "9fc379c97cfe4e15bba95fe8e63a553e"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "7e7742afcc2f1fa8655184b3571d558b"
server_key_iv: "fd8bc410d303afa30c49c4b8f75b335a"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.486241 18071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "9fc379c97cfe4e15bba95fe8e63a553e"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "7e7742afcc2f1fa8655184b3571d558b"
server_key_iv: "fd8bc410d303afa30c49c4b8f75b335a"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.487691 18071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:11:28.488418 18086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:28.488585 18071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:28.488668 18071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "9fc379c97cfe4e15bba95fe8e63a553e"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "7e7742afcc2f1fa8655184b3571d558b"
server_key_iv: "fd8bc410d303afa30c49c4b8f75b335a"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.488729 18071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:28.526638 18071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:28.528467 18071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:28.528609 18071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:28.532299 18071 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:38811
I20260927 07:11:28.532351 18138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:38811 every 8 connection(s)
I20260927 07:11:28.532668 18071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:11:28.533499 18139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:28.535055 18139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:28.542481 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 18071
I20260927 07:11:28.542610 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:11:28.542793 4793 external_mini_cluster.cc:1428] Setting key 545d6885e60535824f7bae997d377fa1
I20260927 07:11:28.544001 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.535679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38811 (local address 127.0.0.1:55558)
0927 07:11:28.535829 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.535864 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.536324 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.536590 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.536593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.536843 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.537001 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.537006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.537989 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.537990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.538701 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.538704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.539144 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.539145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.539218 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.539643 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.539651 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.541027 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.542639 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.542646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.542648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.542840 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.543075 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.543076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.543076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.543154 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.543465 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.543469 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.543531 (+ 62us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.543610 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.543820 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":78,"thread_start_us":62,"threads_started":1}
I20260927 07:11:28.543999 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.535827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.4.174.126:38811)
0927 07:11:28.535924 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.535928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.535959 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:11:28.536479 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.536481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.536497 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.536572 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.537088 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.537898 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.538771 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.539074 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.539168 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.541105 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.541112 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.541113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.541120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.542502 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.542938 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.542941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.542941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.543017 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.543203 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.543204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.543204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.543404 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.543475 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.543648 (+ 173us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.543754 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":38,"threads_started":1}
I20260927 07:11:28.545122 18139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:11:28.545444 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.545332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33462)
0927 07:11:28.545371 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.545423 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260927 07:11:28.545555 18090 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
I20260927 07:11:28.545900 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.545819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33476)
0927 07:11:28.545854 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.545878 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260927 07:11:28.545997 18139 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:46781: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.174.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab.
Entry for principal kudu/127.4.174.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab.
I20260927 07:11:28.565423 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.125:46781
--webserver_interface=127.4.174.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab
--principal=kudu/127.4.174.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:38811,127.4.174.125:46781,127.4.174.124:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:11:28.595189 18139 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } attempt: 1
I20260927 07:11:28.595671 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.595555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33492)
0927 07:11:28.595609 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.595641 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 07:11:28.595980 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.595898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33502)
0927 07:11:28.595939 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.595962 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260927 07:11:28.596071 18139 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:46781: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
W20260927 07:11:28.644017 18146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:28.644248 18146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:28.644281 18146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:28.645408 18146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:28.645447 18146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:28.645465 18146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:28.645480 18146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:28.645495 18146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:28.647012 18146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:38811,127.4.174.125:46781,127.4.174.124:37897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab
--principal=kudu/127.4.174.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.125:46781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.174.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:28.647251 18146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:28.647455 18146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:28.652176 18152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:28.652146 18151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:28.652146 18154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:28.652444 18146 server_base.cc:1034] running on GCE node
I20260927 07:11:28.652685 18146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:28.652989 18146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:28.654151 18146 hybrid_clock.cc:648] HybridClock initialized: now 1790493088654130 us; error 35 us; skew 500 ppm
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:28.656617 18146 init.cc:377] Logged in from keytab as kudu/127.4.174.125@KRBTEST.COM (short username kudu)
I20260927 07:11:28.657053 18146 webserver.cc:460] Webserver started at http://127.4.174.125:40855/ using document root <none> and password file <none>
I20260927 07:11:28.657250 18146 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:28.657289 18146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:28.657385 18146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:28.658393 18146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/data/instance:
uuid: "9f5b947beb0246f6a21427c7e83f15f2"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "81780fe0adcac5dd2ad0dd8f39321441"
server_key_iv: "e9c814bfbcda21ffcc8aa49e410b5979"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.658731 18146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/wal/instance:
uuid: "9f5b947beb0246f6a21427c7e83f15f2"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "81780fe0adcac5dd2ad0dd8f39321441"
server_key_iv: "e9c814bfbcda21ffcc8aa49e410b5979"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.659909 18146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:11:28.660535 18161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:28.660719 18146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:11:28.660815 18146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/wal
uuid: "9f5b947beb0246f6a21427c7e83f15f2"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "81780fe0adcac5dd2ad0dd8f39321441"
server_key_iv: "e9c814bfbcda21ffcc8aa49e410b5979"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.660880 18146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:28.664191 18139 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } attempt: 2
I20260927 07:11:28.664561 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.664447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33512)
0927 07:11:28.664499 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.664531 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 07:11:28.664912 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.664809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33522)
0927 07:11:28.664866 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.664892 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 07:11:28.664997 18139 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:46781: Network error: Client connection negotiation failed: client connection to 127.4.174.125:46781: connect: Connection refused (error 111)
I20260927 07:11:28.685699 18146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:28.687736 18146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:28.687846 18146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:28.692126 18146 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.125:46781
I20260927 07:11:28.692154 18213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.125:46781 every 8 connection(s)
I20260927 07:11:28.692556 18146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
I20260927 07:11:28.693398 18214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:28.694972 18214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:11:28.700132 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 18146
I20260927 07:11:28.700222 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-1/wal/instance
I20260927 07:11:28.700385 4793 external_mini_cluster.cc:1428] Setting key ab5225ca87e0eff700faf7a513183e6b
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:28.703519 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.695649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.4.174.126:38811)
0927 07:11:28.695702 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.695704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.695909 (+ 205us) server_negotiation.cc:407] Connection header received
0927 07:11:28.696436 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.696437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.696453 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.696497 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.697110 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.697614 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.698616 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.698821 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.698878 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.700916 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.700918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.700918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.700926 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.702227 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.702630 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.702632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.702632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.702689 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.702909 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.702910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.702910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.703020 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.703076 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.703333 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.703430 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 07:11:28.703681 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.695620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38811 (local address 127.0.0.1:55570)
0927 07:11:28.695791 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.695828 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.696325 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.696577 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.696581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.696822 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.697023 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.697028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.697698 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.697699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.698510 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.698512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.698892 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.698893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.699023 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.699455 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.699463 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.700826 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.702305 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.702313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.702315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.702538 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.702745 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.702746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.702747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.702854 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.703074 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.703081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.703218 (+ 137us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.703306 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.703464 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260927 07:11:28.704171 18214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.174.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.124@KRBTEST.COM" created.
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:11:28.712762 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.704395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33524)
0927 07:11:28.704566 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.704576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.704616 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.704730 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.704731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.704860 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.704903 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.704904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.705715 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.705716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.706211 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.706212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.706668 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.706669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.706732 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.707112 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.707116 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.709956 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.711817 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.711817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.711818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.711969 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.712141 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.712142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.712142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.712194 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.712455 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.712456 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.712494 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.712554 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.712658 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260927 07:11:28.712843 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.704462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33524 (local address 127.4.174.125:46781)
0927 07:11:28.704594 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.704598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.704617 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:11:28.704657 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.704659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.704671 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.704731 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.704970 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.705642 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.706275 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.706609 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.706670 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.710034 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.710045 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.710045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.710052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.711752 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.712035 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.712038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.712038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.712091 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.712242 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.712243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.712243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.712396 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.712465 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.712591 (+ 126us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.712731 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260927 07:11:28.713392 18214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:11:28.713737 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.713600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:43954)
0927 07:11:28.713649 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.713705 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 07:11:28.713874 18163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
I20260927 07:11:28.714257 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.714161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:43962)
0927 07:11:28.714204 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.714233 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260927 07:11:28.714352 18214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37897: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab.
Entry for principal kudu/127.4.174.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab.
I20260927 07:11:28.725349 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.124:37897
--webserver_interface=127.4.174.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab
--principal=kudu/127.4.174.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:38811,127.4.174.125:46781,127.4.174.124:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:11:28.756160 18139 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } attempt: 3
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:11:28.763195 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.756621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.4.174.125:46781)
0927 07:11:28.756680 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.756682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.756722 (+ 40us) server_negotiation.cc:407] Connection header received
0927 07:11:28.756751 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.756752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.756763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.756816 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.757117 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.757937 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.758646 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.758987 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.759058 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.760852 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.760854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.760854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.760860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.762093 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.762492 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.762492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.762493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.762535 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.762755 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.762755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.762756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.762856 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.762900 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.763050 (+ 150us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.763110 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 07:11:28.763221 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.756541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33538)
0927 07:11:28.756604 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.756617 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.756667 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.756826 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.756827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.756989 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.757053 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.757055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.758010 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.758010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.758529 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.758530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.759056 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.759057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.759118 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.759549 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.759553 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.760765 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.762176 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.762177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.762178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.762426 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.762623 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.762624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.762624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.762698 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.762910 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.762911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.762961 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.763031 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.763124 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 07:11:28.763581 18214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } attempt: 1
I20260927 07:11:28.763690 18139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:11:28.763921 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.763799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:43972)
0927 07:11:28.763865 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.763891 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260927 07:11:28.764020 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.763904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:43976)
0927 07:11:28.763967 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.763994 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 07:11:28.764202 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.764131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:43980)
0927 07:11:28.764158 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.764183 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260927 07:11:28.764324 18214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37897: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
I20260927 07:11:28.764425 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.764332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:43982)
0927 07:11:28.764378 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.764405 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 07:11:28.764508 18139 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37897: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
I20260927 07:11:28.795639 18139 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } attempt: 1
I20260927 07:11:28.796051 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.795940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:43986)
0927 07:11:28.795993 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.796022 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 07:11:28.796483 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.796371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:43996)
0927 07:11:28.796430 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.796458 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260927 07:11:28.796579 18139 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37897: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
W20260927 07:11:28.808709 18221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:28.808924 18221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:28.808954 18221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:28.810070 18221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:28.810111 18221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:28.810130 18221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:28.810150 18221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:28.810166 18221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:28.811666 18221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:38811,127.4.174.125:46781,127.4.174.124:37897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab
--principal=kudu/127.4.174.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.124:37897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.174.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:28.811904 18221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:28.812103 18221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:28.816668 18229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:28.816691 18226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:28.816668 18227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:28.816692 18221 server_base.cc:1034] running on GCE node
I20260927 07:11:28.817051 18221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:28.817318 18221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:28.818468 18221 hybrid_clock.cc:648] HybridClock initialized: now 1790493088818455 us; error 29 us; skew 500 ppm
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:28.820874 18221 init.cc:377] Logged in from keytab as kudu/127.4.174.124@KRBTEST.COM (short username kudu)
I20260927 07:11:28.821321 18221 webserver.cc:460] Webserver started at http://127.4.174.124:38829/ using document root <none> and password file <none>
I20260927 07:11:28.821527 18221 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:28.821575 18221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:28.821699 18221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:28.822795 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/data/instance:
uuid: "30e67667d3c94209ad89fb48ecd8cf2c"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "c9af01225081521a72a807aefcb740e9"
server_key_iv: "3e695582316b4fb790ec5670a7a2741a"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.823129 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/wal/instance:
uuid: "30e67667d3c94209ad89fb48ecd8cf2c"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "c9af01225081521a72a807aefcb740e9"
server_key_iv: "3e695582316b4fb790ec5670a7a2741a"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.824366 18221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:11:28.825059 18236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:28.825217 18221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:28.825296 18221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/wal
uuid: "30e67667d3c94209ad89fb48ecd8cf2c"
format_stamp: "Formatted at 2026-09-27 07:11:28 on dist-test-slave-zw7p"
server_key: "c9af01225081521a72a807aefcb740e9"
server_key_iv: "3e695582316b4fb790ec5670a7a2741a"
server_key_version: "encryptionkey@0"
I20260927 07:11:28.825354 18221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:28.832468 18214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } attempt: 2
I20260927 07:11:28.832858 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.832720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:44002)
0927 07:11:28.832775 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.832816 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 07:11:28.833163 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.833084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:44012)
0927 07:11:28.833120 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.833144 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260927 07:11:28.833244 18214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37897: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37897: connect: Connection refused (error 111)
I20260927 07:11:28.838837 18221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:28.840780 18221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:28.840905 18221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:28.844632 18221 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.124:37897
I20260927 07:11:28.844660 18288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.124:37897 every 8 connection(s)
I20260927 07:11:28.844982 18221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
I20260927 07:11:28.845808 18289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:28.847371 18289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:11:28.850154 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 18221
I20260927 07:11:28.850229 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492940996973-4793-0/minicluster-data/master-2/wal/instance
I20260927 07:11:28.850379 4793 external_mini_cluster.cc:1428] Setting key e3852b087aab783058822d84d69d6ac3
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:11:28.856271 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.848102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55574 (local address 127.4.174.126:38811)
0927 07:11:28.848181 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.848184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.848334 (+ 150us) server_negotiation.cc:407] Connection header received
0927 07:11:28.848821 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.848823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.848835 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.848893 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.849497 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.849983 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.851160 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.851361 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.851417 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.853348 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.853350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.853350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.853357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.854825 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.855192 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.855193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.855194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.855246 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.855538 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.855539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.855539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.855695 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.855761 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.856091 (+ 330us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.856179 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 07:11:28.856436 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.848029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:38811 (local address 127.0.0.1:55574)
0927 07:11:28.848221 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.848258 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.848725 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.848961 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.848964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.849207 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.849401 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.849405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.850064 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.850065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.851084 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.851087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.851427 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.851427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.851565 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.852016 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.852024 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.853267 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.854898 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.854904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.854906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.855120 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.855302 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.855303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.855303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.855455 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.855752 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.855758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.855936 (+ 178us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.856050 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.856198 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260927 07:11:28.856969 18289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:11:28.858294 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.850729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.4.174.124:37897)
0927 07:11:28.850883 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.850889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.850911 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:11:28.850957 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.850959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.850978 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.851040 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.851587 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.852261 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.852946 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.853272 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.853328 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.855561 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.855567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.855568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.855573 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.857244 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.857556 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.857559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.857559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.857628 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.857792 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.857793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.857793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.857924 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.858063 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.858129 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.858200 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:28.863170 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.857274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.4.174.125:46781)
0927 07:11:28.857340 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.857341 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.857374 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:11:28.857436 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.857437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.857446 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.857491 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.857708 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.858151 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.858953 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.859232 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.859283 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.860854 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.860855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.860856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.860863 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.862166 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.862494 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.862495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.862496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.862548 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.862758 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.862758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.862758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.862853 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.862913 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.863053 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.863111 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 07:11:28.863204 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.857199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:46781 (local address 127.0.0.1:33552)
0927 07:11:28.857340 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.857350 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.857385 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.857499 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.857499 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.857621 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.857646 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.857647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.858209 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.858210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.858872 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.858875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.859300 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.859301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.859365 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.859684 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.859688 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.860783 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.862242 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.862243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.862243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.862429 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.862635 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.862636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.862636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.862700 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.862916 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.862917 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.862957 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.863034 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.863117 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260927 07:11:28.863611 18289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:11:28.865662 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.859011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55588 (local address 127.4.174.126:38811)
0927 07:11:28.859044 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.859046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.859052 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:11:28.859084 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.859085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.859093 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.859137 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.859608 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.860090 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.860798 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.861045 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.861112 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.863307 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.863309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.863309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.863316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.864750 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.865077 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.865078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.865078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.865121 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.865261 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.865261 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.865262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.865344 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.865465 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.865501 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.865573 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:11:28.869969 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.863814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:44042)
0927 07:11:28.863866 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.863877 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.863925 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.864038 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.864039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.864148 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.864169 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.864171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.864770 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.864771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.865446 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.865448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.865985 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.865985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.866021 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.866335 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.866340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.867607 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.869133 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.869134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.869134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.869277 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.869425 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.869426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.869426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.869475 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.869660 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.869661 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.869695 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.869749 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.869835 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 07:11:28.870015 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.863857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.4.174.124:37897)
0927 07:11:28.863886 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.863888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.863911 (+ 23us) server_negotiation.cc:407] Connection header received
0927 07:11:28.863976 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.863976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.863983 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.864026 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.864225 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.864710 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.865520 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.865798 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.865858 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.867698 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.867699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.867699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.867705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.869071 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.869328 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.869329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.869329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.869370 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.869521 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.869522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.869522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.869611 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.869661 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.869783 (+ 122us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.869878 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":14}
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:11:28.871452 18289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c: Bootstrap starting.
I20260927 07:11:28.871694 18139 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } attempt: 2
I20260927 07:11:28.872462 18289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:28.872753 18289 log.cc:824] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:28.873500 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.866345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.4.174.125:46781)
0927 07:11:28.866381 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.866383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.866388 (+ 5us) server_negotiation.cc:407] Connection header received
0927 07:11:28.866403 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.866404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.866410 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.866449 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.867005 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.867601 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.868501 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.868771 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.868867 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.870941 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.870942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.870943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.870949 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.872365 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.872717 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.872719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.872719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.872778 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.873003 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.873004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.873004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.873127 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.873304 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.873341 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.873411 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:11:28.874333 18289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c: No bootstrap required, opened a new log
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:11:28.876147 18289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:28.876272 18289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:28.876305 18289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e67667d3c94209ad89fb48ecd8cf2c, State: Initialized, Role: FOLLOWER
I20260927 07:11:28.876421 18289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:28.876659 18295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:28.876715 18289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:28.876739 18295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:11:28.878187 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.871913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:44044)
0927 07:11:28.871972 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.871985 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.872032 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.872151 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.872152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.872298 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.872339 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.872341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.873073 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.873074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.873769 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.873772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.874178 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.874179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.874239 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.874603 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.874608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.875737 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.877096 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.877097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.877097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.877307 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.877490 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.877491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.877491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.877550 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.877896 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.877897 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.877943 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.878010 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.878103 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 07:11:28.878646 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.872004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.4.174.124:37897)
0927 07:11:28.872035 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.872037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.872045 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:11:28.872084 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.872085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.872093 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.872147 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.872407 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.873014 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.873853 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.874108 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.874159 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.875810 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.875812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.875812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.875818 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.877020 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.877382 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.877383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.877383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.877435 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.877646 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.877647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.877647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.877791 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.877859 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.878478 (+ 619us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.878547 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260927 07:11:28.878677 18306 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:11:28.878711 18306 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:11:28.879096 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:11:28.880157 18139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: Bootstrap starting.
I20260927 07:11:28.880872 18139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:28.878868 18221 master_runner.cc:186] Error getting master registration for 127.4.174.126:38811: OK, instance_id { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" instance_seqno: 1790493088531267 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:11:28.881057 18221 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:11:28.881183 18139 log.cc:824] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:28.881842 18139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: No bootstrap required, opened a new log
I20260927 07:11:28.883484 18139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:28.883591 18139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:28.883622 18139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc379c97cfe4e15bba95fe8e63a553e, State: Initialized, Role: FOLLOWER
I20260927 07:11:28.883742 18139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:28.883971 18316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:28.884032 18139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:28.884099 18316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [sys.catalog]: This master's current role is: FOLLOWER
W20260927 07:11:28.885614 18327 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:11:28.885793 18327 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:11:28.886548 18071 master_runner.cc:186] Error getting master registration for 127.4.174.125:46781: OK, instance_id { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" instance_seqno: 1790493088690860 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:11:28.889936 18071 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:11:28.892738 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.881046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44060 (local address 127.4.174.124:37897)
0927 07:11:28.881152 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.881154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.883858 (+ 2704us) server_negotiation.cc:407] Connection header received
0927 07:11:28.883938 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.883939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.883949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.883974 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.885344 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.886028 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.887450 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.887742 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.887804 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.889908 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.889909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.889910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.889917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.891443 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.891847 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.891848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.891848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.891903 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.892076 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.892077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.892077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.892194 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.892270 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.892563 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.892645 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 07:11:28.893802 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.880516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.4.174.126:38811)
0927 07:11:28.880555 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.880557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.881213 (+ 656us) server_negotiation.cc:407] Connection header received
0927 07:11:28.881299 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.881300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.881310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.881337 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.882036 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.882786 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.884674 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.884963 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.885003 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.887050 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.887051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.887052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.887058 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.889008 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.892955 (+ 3947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.892956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.892956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.893018 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.893218 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.893219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.893220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.893346 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.893437 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.893640 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.893716 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:11:28.894495 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.882776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33564 (local address 127.4.174.125:46781)
0927 07:11:28.882817 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.882819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.887547 (+ 4728us) server_negotiation.cc:407] Connection header received
0927 07:11:28.887634 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.887635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.887646 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.887672 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.888247 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.888926 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.889833 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.890084 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.890126 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.891494 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.891495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.891495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.891502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.892954 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.893517 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.893518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.893519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.893575 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.893765 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.893765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.893766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.893884 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.893951 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.894317 (+ 366us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.894403 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:11:28.924837 18214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } attempt: 3
Sep 27 07:11:28 dist-test-slave-zw7p krb5kdc[18058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493088, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:11:28.932967 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.925220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37897 (local address 127.0.0.1:44066)
0927 07:11:28.925461 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:28.925475 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:28.925539 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:28.925741 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:28.925742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:28.925930 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:28.925984 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.925987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.926930 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.926932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.927724 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:28.927727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.928217 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.928218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:28.928275 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.928764 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:28.928770 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:28.930233 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:28.931956 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.931957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.931958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.932157 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.932341 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:28.932342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:28.932342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:28.932408 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:28.932671 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:28.932672 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:28.932714 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 07:11:28.932771 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 07:11:28.932863 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260927 07:11:28.933312 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:28.925329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44066 (local address 127.4.174.124:37897)
0927 07:11:28.925614 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:28.925616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:28.925623 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:11:28.925645 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:28.925646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:28.925656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:28.925697 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:28.926063 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.926838 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.927815 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:28.928099 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:28.928158 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:28.930321 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:28.930323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:28.930323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:28.930330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:28.931867 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.932228 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.932229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.932229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.932281 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:28.932469 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:28.932470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:28.932470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:28.932594 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:28.932647 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:28.933152 (+ 505us) server_negotiation.cc:299] Negotiation successful
0927 07:11:28.933226 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260927 07:11:28.935248 18214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2: Bootstrap starting.
I20260927 07:11:28.936014 18214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:28.936319 18214 log.cc:824] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:28.937512 18214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2: No bootstrap required, opened a new log
I20260927 07:11:28.939556 18214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:28.939693 18214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:28.939728 18214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5b947beb0246f6a21427c7e83f15f2, State: Initialized, Role: FOLLOWER
I20260927 07:11:28.939848 18214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:28.940163 18214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:28.940126 18330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:28.940307 18330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 07:11:28.941888 18341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:11:28.941984 18341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:11:28.944245 18146 master_runner.cc:263] No leader master found from master 9f5b947beb0246f6a21427c7e83f15f2
I20260927 07:11:28.944294 18146 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260927 07:11:29.029385 18092 tablet.cc:2369] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:11:29.081161 18295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:29.081298 18295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:29.081547 18295 leader_election.cc:288] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fc379c97cfe4e15bba95fe8e63a553e (127.4.174.126:38811), 9f5b947beb0246f6a21427c7e83f15f2 (127.4.174.125:46781)
I20260927 07:11:29.081790 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" is_pre_election: true
I20260927 07:11:29.081851 18188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5b947beb0246f6a21427c7e83f15f2" is_pre_election: true
I20260927 07:11:29.081964 18113 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e67667d3c94209ad89fb48ecd8cf2c in term 0.
I20260927 07:11:29.081972 18188 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e67667d3c94209ad89fb48ecd8cf2c in term 0.
I20260927 07:11:29.082158 18239 leader_election.cc:302] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30e67667d3c94209ad89fb48ecd8cf2c, 9fc379c97cfe4e15bba95fe8e63a553e; no voters:
I20260927 07:11:29.082370 18295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:11:29.082454 18295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:11:29.082484 18295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:29.083256 18295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:29.083400 18295 leader_election.cc:288] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [CANDIDATE]: Term 1 election: Requested vote from peers 9fc379c97cfe4e15bba95fe8e63a553e (127.4.174.126:38811), 9f5b947beb0246f6a21427c7e83f15f2 (127.4.174.125:46781)
I20260927 07:11:29.083554 18188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5b947beb0246f6a21427c7e83f15f2"
I20260927 07:11:29.083643 18188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:29.083714 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc379c97cfe4e15bba95fe8e63a553e"
I20260927 07:11:29.083810 18113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:29.084766 18113 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e67667d3c94209ad89fb48ecd8cf2c in term 1.
I20260927 07:11:29.084930 18188 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e67667d3c94209ad89fb48ecd8cf2c in term 1.
I20260927 07:11:29.084952 18239 leader_election.cc:302] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30e67667d3c94209ad89fb48ecd8cf2c, 9fc379c97cfe4e15bba95fe8e63a553e; no voters:
I20260927 07:11:29.085063 18295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:11:29.085256 18295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [term 1 LEADER]: Becoming Leader. State: Replica: 30e67667d3c94209ad89fb48ecd8cf2c, State: Running, Role: LEADER
I20260927 07:11:29.085383 18295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } }
I20260927 07:11:29.085806 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30e67667d3c94209ad89fb48ecd8cf2c. Latest consensus state: current_term: 1 leader_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:29.085913 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:29.086123 18345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:11:29.086308 18345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:11:29.087731 18188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [term 1 FOLLOWER]: Refusing update from remote peer 30e67667d3c94209ad89fb48ecd8cf2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:29.087764 18113 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [term 1 FOLLOWER]: Refusing update from remote peer 30e67667d3c94209ad89fb48ecd8cf2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:29.088075 18343 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:29.088213 18295 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:29.089439 18330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30e67667d3c94209ad89fb48ecd8cf2c. Latest consensus state: current_term: 1 leader_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:29.089447 18316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30e67667d3c94209ad89fb48ecd8cf2c. Latest consensus state: current_term: 1 leader_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:29.089536 18316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:11:29.089538 18330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:11:29.090458 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:29.090540 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:29.090654 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:29.090711 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:29.090765 18346 mvcc.cc:205] Tried to move back new op lower bound from 7333859692901269504 to 7333859692894388224. Current Snapshot: MvccSnapshot[applied={T|T < 7333859692901269504}]
I20260927 07:11:29.091015 18330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:29.091110 18330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:11:29.091183 18316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc379c97cfe4e15bba95fe8e63a553e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 38811 } } peers { permanent_uuid: "9f5b947beb0246f6a21427c7e83f15f2" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 46781 } } peers { permanent_uuid: "30e67667d3c94209ad89fb48ecd8cf2c" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37897 } } }
I20260927 07:11:29.091253 18316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:11:29.091590 18347 mvcc.cc:205] Tried to move back new op lower bound from 7333859692901269504 to 7333859692894388224. Current Snapshot: MvccSnapshot[applied={T|T < 7333859692901269504}]
I20260927 07:11:29.091516 18348 mvcc.cc:205] Tried to move back new op lower bound from 7333859692901269504 to 7333859692894388224. Current Snapshot: MvccSnapshot[applied={T|T < 7333859692901269504 or (T in {7333859692901269504})}]
I20260927 07:11:29.091758 18345 catalog_manager.cc:1347] Generated new cluster ID: 16372b8adfb5495e9ed70791594eb7ba
I20260927 07:11:29.091805 18345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:11:29.141360 18345 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:11:29.142016 18345 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:11:29.147454 18345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30e67667d3c94209ad89fb48ecd8cf2c: Generated new TSK 0
I20260927 07:11:29.147653 18345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:11:29.886272 18327 catalog_manager.cc:1259] Loaded cluster ID: 16372b8adfb5495e9ed70791594eb7ba
I20260927 07:11:29.886348 18327 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: loading cluster ID for follower catalog manager: success
I20260927 07:11:29.887188 18327 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: acquiring CA information for follower catalog manager: success
I20260927 07:11:29.887498 18327 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9fc379c97cfe4e15bba95fe8e63a553e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:11:29.942449 18341 catalog_manager.cc:1259] Loaded cluster ID: 16372b8adfb5495e9ed70791594eb7ba
I20260927 07:11:29.942538 18341 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2: loading cluster ID for follower catalog manager: success
I20260927 07:11:29.943362 18341 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2: acquiring CA information for follower catalog manager: success
I20260927 07:11:29.943671 18341 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f5b947beb0246f6a21427c7e83f15f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:11:30.204582 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 18071
I20260927 07:11:30.209408 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 18146
I20260927 07:11:30.213725 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 18221
2026-09-27T07:11:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2889 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:11:30 dist-test-slave-zw7p krb5kdc[18360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:11:30 dist-test-slave-zw7p krb5kdc[18360](info): set up 2 sockets
Sep 27 07:11:30 dist-test-slave-zw7p krb5kdc[18360](info): commencing operation
krb5kdc: starting...
W20260927 07:11:31.178915 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:11:31 dist-test-slave-zw7p krb5kdc[18360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:11:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:11:31.293421 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:41495
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:38323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:41495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:31.376585 18373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:31.376796 18373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:31.376824 18373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:31.377982 18373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:31.378022 18373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:31.378041 18373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:31.378057 18373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:31.378072 18373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:31.379542 18373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:38323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:41495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:41495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:31.379820 18373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:31.380028 18373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:31.384857 18378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:31.384891 18379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:31.386933 18373 server_base.cc:1034] running on GCE node
W20260927 07:11:31.387140 18381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:31.387439 18373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:31.387709 18373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:31.388917 18373 hybrid_clock.cc:648] HybridClock initialized: now 1790493091388900 us; error 29 us; skew 500 ppm
Sep 27 07:11:31 dist-test-slave-zw7p krb5kdc[18360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493091, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:31.391515 18373 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:11:31.391973 18373 webserver.cc:460] Webserver started at http://127.4.174.126:44127/ using document root <none> and password file <none>
I20260927 07:11:31.392184 18373 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:31.392235 18373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:31.392369 18373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:31.393499 18373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "6c574dd78b694e4085d5370152d5dda8"
format_stamp: "Formatted at 2026-09-27 07:11:31 on dist-test-slave-zw7p"
server_key: "0522fdcd7de2df1c0be029e0db6f348d"
server_key_iv: "dbaa34d0c16764334b4b9f8f25250a05"
server_key_version: "encryptionkey@0"
I20260927 07:11:31.393893 18373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "6c574dd78b694e4085d5370152d5dda8"
format_stamp: "Formatted at 2026-09-27 07:11:31 on dist-test-slave-zw7p"
server_key: "0522fdcd7de2df1c0be029e0db6f348d"
server_key_iv: "dbaa34d0c16764334b4b9f8f25250a05"
server_key_version: "encryptionkey@0"
I20260927 07:11:31.395428 18373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:11:31.396137 18388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:31.396289 18373 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:11:31.396373 18373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "6c574dd78b694e4085d5370152d5dda8"
format_stamp: "Formatted at 2026-09-27 07:11:31 on dist-test-slave-zw7p"
server_key: "0522fdcd7de2df1c0be029e0db6f348d"
server_key_iv: "dbaa34d0c16764334b4b9f8f25250a05"
server_key_version: "encryptionkey@0"
I20260927 07:11:31.396443 18373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:31.405649 18373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:31.407600 18373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:31.407742 18373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:31.412082 18373 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:41495
I20260927 07:11:31.412118 18440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:41495 every 8 connection(s)
I20260927 07:11:31.412467 18373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:11:31.413306 18441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:31.415683 18441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8: Bootstrap starting.
I20260927 07:11:31.416235 18441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:31.416548 18441 log.cc:824] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:31.417171 18441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8: No bootstrap required, opened a new log
I20260927 07:11:31.417822 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 18373
I20260927 07:11:31.417924 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:11:31.418084 4793 external_mini_cluster.cc:1428] Setting key 2f08d7e757c8f53621ca03caf1451ea7
I20260927 07:11:31.419013 18441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c574dd78b694e4085d5370152d5dda8" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41495 } }
I20260927 07:11:31.419155 18441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:31.419188 18441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c574dd78b694e4085d5370152d5dda8, State: Initialized, Role: FOLLOWER
I20260927 07:11:31.419337 18441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c574dd78b694e4085d5370152d5dda8" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41495 } }
I20260927 07:11:31.419436 18441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:11:31.419476 18441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:11:31.419526 18441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:31.420601 18441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c574dd78b694e4085d5370152d5dda8" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41495 } }
I20260927 07:11:31.420692 18441 leader_election.cc:302] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c574dd78b694e4085d5370152d5dda8; no voters:
I20260927 07:11:31.420837 18441 leader_election.cc:288] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:11:31.420871 18446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:11:31.420976 18446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [term 1 LEADER]: Becoming Leader. State: Replica: 6c574dd78b694e4085d5370152d5dda8, State: Running, Role: LEADER
I20260927 07:11:31.421094 18441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:31.421075 18446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c574dd78b694e4085d5370152d5dda8" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41495 } }
I20260927 07:11:31.421398 18446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c574dd78b694e4085d5370152d5dda8. Latest consensus state: current_term: 1 leader_uuid: "6c574dd78b694e4085d5370152d5dda8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c574dd78b694e4085d5370152d5dda8" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41495 } } }
I20260927 07:11:31.421478 18446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:31.421412 18447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c574dd78b694e4085d5370152d5dda8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c574dd78b694e4085d5370152d5dda8" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41495 } } }
I20260927 07:11:31.421626 18447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8 [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:31.422425 18452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:11:31.423050 18452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:11:31.424580 18452 catalog_manager.cc:1347] Generated new cluster ID: bf2e69b59cfb4840bf7df2d83e763282
I20260927 07:11:31.424623 18452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:11:31 dist-test-slave-zw7p krb5kdc[18360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:31.429749 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:31.418532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.4.174.126:41495)
0927 07:11:31.418729 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:31.418738 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:31.418763 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:11:31.418988 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:31.418994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:31.419039 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:31.419151 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:31.420084 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:31.421476 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:31.422547 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:31.423161 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:31.423261 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:31.425862 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:31.425866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:31.425867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:31.425878 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:31.427731 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:31.428134 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:31.428138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:31.428138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:31.428264 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:31.428442 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:31.428443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:31.428443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:31.428747 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:31.429314 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:31.429394 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 07:11:31.429542 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260927 07:11:31.430689 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:11:31.434531 18452 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:11:31.435034 18452 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:11:31.437625 18452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c574dd78b694e4085d5370152d5dda8: Generated new TSK 0
I20260927 07:11:31.437785 18452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:11:31.484899 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 18373
2026-09-27T07:11:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1269 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:11:31 dist-test-slave-zw7p krb5kdc[18468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:11:31 dist-test-slave-zw7p krb5kdc[18468](info): set up 2 sockets
Sep 27 07:11:31 dist-test-slave-zw7p krb5kdc[18468](info): commencing operation
krb5kdc: starting...
W20260927 07:11:32.426280 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:11:32 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:11:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:11:32.539295 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40505
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:32.619016 18481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:32.619225 18481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:32.619254 18481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:32.620436 18481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:32.620487 18481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:32.620507 18481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:32.620522 18481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:32.620536 18481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:32.622061 18481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:32.622315 18481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:32.622521 18481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:32.627182 18487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:32.627187 18486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:32.627204 18489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:32.627434 18481 server_base.cc:1034] running on GCE node
I20260927 07:11:32.627645 18481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:32.627918 18481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:32.629065 18481 hybrid_clock.cc:648] HybridClock initialized: now 1790493092629054 us; error 32 us; skew 500 ppm
Sep 27 07:11:32 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493092, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:32.631461 18481 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:11:32.631870 18481 webserver.cc:460] Webserver started at http://127.4.174.126:40655/ using document root <none> and password file <none>
I20260927 07:11:32.632068 18481 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:32.632112 18481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:32.632215 18481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:32.633268 18481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "7c172b93c2bc4db8bb48da90bcf3eecf"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "51dbfd0a7d0f100fcf2b307b0c0a1a19"
server_key_iv: "f2ba00e42ff143f20e12ea8b15d9e2d7"
server_key_version: "encryptionkey@0"
I20260927 07:11:32.633634 18481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "7c172b93c2bc4db8bb48da90bcf3eecf"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "51dbfd0a7d0f100fcf2b307b0c0a1a19"
server_key_iv: "f2ba00e42ff143f20e12ea8b15d9e2d7"
server_key_version: "encryptionkey@0"
I20260927 07:11:32.634888 18481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:11:32.635562 18496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:32.635699 18481 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:11:32.635773 18481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "7c172b93c2bc4db8bb48da90bcf3eecf"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "51dbfd0a7d0f100fcf2b307b0c0a1a19"
server_key_iv: "f2ba00e42ff143f20e12ea8b15d9e2d7"
server_key_version: "encryptionkey@0"
I20260927 07:11:32.635835 18481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:32.656525 18481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:32.658602 18481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:32.658716 18481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:32.662334 18481 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40505
I20260927 07:11:32.662379 18548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40505 every 8 connection(s)
I20260927 07:11:32.662711 18481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:11:32.663643 18549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:32.664101 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 18481
I20260927 07:11:32.664181 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:11:32.664351 4793 external_mini_cluster.cc:1428] Setting key 7bf1d72057253a25e5011a5126203033
I20260927 07:11:32.666368 18549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Bootstrap starting.
I20260927 07:11:32.667068 18549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:32.667385 18549 log.cc:824] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:32.668036 18549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: No bootstrap required, opened a new log
I20260927 07:11:32.670022 18549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:32.670156 18549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:32.670186 18549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c172b93c2bc4db8bb48da90bcf3eecf, State: Initialized, Role: FOLLOWER
I20260927 07:11:32.670307 18549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:32.670398 18549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:11:32.670431 18549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:11:32.670480 18549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 0 FOLLOWER]: Advancing to term 1
Sep 27 07:11:32 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:32.670998 18549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:32.671097 18549 leader_election.cc:302] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c172b93c2bc4db8bb48da90bcf3eecf; no voters:
I20260927 07:11:32.671244 18549 leader_election.cc:288] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:11:32.671274 18554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:11:32.671424 18554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 1 LEADER]: Becoming Leader. State: Replica: 7c172b93c2bc4db8bb48da90bcf3eecf, State: Running, Role: LEADER
I20260927 07:11:32.671511 18549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:32.671567 18554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:32.671904 18554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c172b93c2bc4db8bb48da90bcf3eecf. Latest consensus state: current_term: 1 leader_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } } }
I20260927 07:11:32.671937 18555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } } }
I20260927 07:11:32.672148 18555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:32.671980 18554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:32.672789 18563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:11:32.673362 18563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:11:32.674927 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:32.664876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34324 (local address 127.4.174.126:40505)
0927 07:11:32.665054 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:32.665064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:32.665102 (+ 38us) server_negotiation.cc:407] Connection header received
0927 07:11:32.665599 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:32.665608 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:32.665672 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:32.665834 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:32.666416 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:32.667740 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:32.668477 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:32.669093 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:32.669207 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:32.671417 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:32.671422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:32.671422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:32.671435 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:32.673360 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:32.673692 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:32.673694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:32.673695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:32.673795 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:32.673967 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:32.673968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:32.673968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:32.674199 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:32.674334 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:32.674644 (+ 310us) server_negotiation.cc:299] Negotiation successful
0927 07:11:32.674757 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260927 07:11:32.675064 18563 catalog_manager.cc:1347] Generated new cluster ID: 3d1ee5314a0c43949f829d433e23e04a
I20260927 07:11:32.675099 18563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:11:32.701586 18563 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:11:32.702145 18563 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:11:32.714867 18563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Generated new TSK 0
I20260927 07:11:32.715068 18563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:11:32.749682 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40505
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:32.846084 18575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:32.846287 18575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:32.846310 18575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:32.847470 18575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:32.847510 18575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:32.847535 18575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:11:32.848924 18575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:32.849109 18575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:32.849287 18575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:32.854063 18583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:32.854063 18581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:32.854094 18580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:32.854076 18575 server_base.cc:1034] running on GCE node
I20260927 07:11:32.854481 18575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:32.854722 18575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:32.855863 18575 hybrid_clock.cc:648] HybridClock initialized: now 1790493092855846 us; error 32 us; skew 500 ppm
Sep 27 07:11:32 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493092, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:32.858172 18575 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:11:32.858595 18575 webserver.cc:460] Webserver started at http://127.4.174.65:42347/ using document root <none> and password file <none>
I20260927 07:11:32.858801 18575 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:32.858845 18575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:32.858964 18575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:32.859941 18575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "eceb0933a873433794d22e355afa18b0"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "3aa68033ebdf92fc77dcbde9325b9b46"
server_key_iv: "ad54dc48a4ae81c3423ed9f4cb4601f5"
server_key_version: "encryptionkey@0"
I20260927 07:11:32.860247 18575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "eceb0933a873433794d22e355afa18b0"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "3aa68033ebdf92fc77dcbde9325b9b46"
server_key_iv: "ad54dc48a4ae81c3423ed9f4cb4601f5"
server_key_version: "encryptionkey@0"
I20260927 07:11:32.861413 18575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:11:32.862033 18590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:32.862181 18575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:32.862262 18575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "eceb0933a873433794d22e355afa18b0"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "3aa68033ebdf92fc77dcbde9325b9b46"
server_key_iv: "ad54dc48a4ae81c3423ed9f4cb4601f5"
server_key_version: "encryptionkey@0"
I20260927 07:11:32.862322 18575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:32.874228 18575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:32.876303 18575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:32.876441 18575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:32.876633 18575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:32.876914 18575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:32.876947 18575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:32.876977 18575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:32.877002 18575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:32.882450 18575 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:45371
I20260927 07:11:32.882507 18703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:45371 every 8 connection(s)
I20260927 07:11:32.882830 18575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:11:32.884799 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 18575
I20260927 07:11:32.884905 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:11:32.885107 4793 external_mini_cluster.cc:1428] Setting key 108caa19c1f5b8d65df697c31871b16c
Sep 27 07:11:32 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493092, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:32.892535 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:32.883471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:34967 (local address 127.4.174.126:40505)
0927 07:11:32.883524 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:32.883526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:32.883737 (+ 211us) server_negotiation.cc:407] Connection header received
0927 07:11:32.884200 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:32.884201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:32.884210 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:32.884269 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:32.885407 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:32.885994 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:32.887198 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:32.887411 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:32.887462 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:32.889908 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:32.889909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:32.889910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:32.889916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:32.891160 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:32.891619 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:32.891620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:32.891620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:32.891667 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:32.891916 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:32.891917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:32.891917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:32.892021 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:32.892066 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:32.892395 (+ 329us) server_negotiation.cc:299] Negotiation successful
0927 07:11:32.892468 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 07:11:32.892807 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:32.883452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:34967)
0927 07:11:32.883614 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:32.883656 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:32.884124 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:32.884324 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:32.884328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:32.885014 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:32.885299 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:32.885307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:32.886083 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:32.886084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:32.887098 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:32.887102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:32.887480 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:32.887481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:32.887621 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:32.888125 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:32.888135 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:32.889817 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:32.891238 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:32.891246 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:32.891248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:32.891548 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:32.891720 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:32.891720 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:32.891721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:32.891858 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:32.892077 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:32.892082 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:32.892265 (+ 183us) client_negotiation.cc:769] Sending connection context
0927 07:11:32.892396 (+ 131us) client_negotiation.cc:240] Negotiation successful
0927 07:11:32.892566 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
I20260927 07:11:32.893414 18704 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40505
I20260927 07:11:32.893519 18704 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:32.893719 18704 heartbeater.cc:511] Master 127.4.174.126:40505 requested a full tablet report, sending...
I20260927 07:11:32.894261 18513 ts_manager.cc:194] Registered new tserver with Master: eceb0933a873433794d22e355afa18b0 (127.4.174.65:45371)
I20260927 07:11:32.894950 18513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:34967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:11:32.910439 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40505
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:32.990095 18709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:32.990293 18709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:32.990315 18709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:32.991465 18709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:32.991508 18709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:32.991532 18709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:11:32.992918 18709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:32.993100 18709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:32.993268 18709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:32.997903 18717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:32.997944 18714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:32.998035 18715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:32.998200 18709 server_base.cc:1034] running on GCE node
I20260927 07:11:32.998406 18709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:32.998654 18709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:32.999807 18709 hybrid_clock.cc:648] HybridClock initialized: now 1790493092999774 us; error 29 us; skew 500 ppm
Sep 27 07:11:32 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493092, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:33.002027 18709 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:11:33.002466 18709 webserver.cc:460] Webserver started at http://127.4.174.66:39659/ using document root <none> and password file <none>
I20260927 07:11:33.002682 18709 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:33.002731 18709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:33.002851 18709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:33.003857 18709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "fc562b57ab3f48008b96bbcb836e4486"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "9c865944a0c3bc87e2dd2d1c515a62fa"
server_key_iv: "26cffcc43b4848d54698b80a0fa277f4"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.004195 18709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "fc562b57ab3f48008b96bbcb836e4486"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "9c865944a0c3bc87e2dd2d1c515a62fa"
server_key_iv: "26cffcc43b4848d54698b80a0fa277f4"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.005396 18709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:11:33.006072 18724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.006230 18709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:33.006320 18709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "fc562b57ab3f48008b96bbcb836e4486"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "9c865944a0c3bc87e2dd2d1c515a62fa"
server_key_iv: "26cffcc43b4848d54698b80a0fa277f4"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.006381 18709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:33.020071 18709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:33.021935 18709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:33.022066 18709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:33.022265 18709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:33.022588 18709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:33.022622 18709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.022652 18709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:33.022670 18709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.028187 18709 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:39353
I20260927 07:11:33.028239 18837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:39353 every 8 connection(s)
I20260927 07:11:33.028533 18709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493092, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:33.035288 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 18709
I20260927 07:11:33.035385 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:11:33.035568 4793 external_mini_cluster.cc:1428] Setting key b6ac736e8ae996adc8f707367b7048d0
I20260927 07:11:33.037314 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.029178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:42733 (local address 127.4.174.126:40505)
0927 07:11:33.029219 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.029221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.029423 (+ 202us) server_negotiation.cc:407] Connection header received
0927 07:11:33.029925 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.029926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.029939 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.029987 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:33.030977 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.031471 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.032444 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.032635 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.032689 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.034756 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:33.034758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:33.034758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:33.034766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:33.036016 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.036470 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.036471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.036472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.036532 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.036733 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.036733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.036734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.036863 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:33.036924 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.037141 (+ 217us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.037225 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:11:33.037511 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.029151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:42733)
0927 07:11:33.029299 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.029342 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.029820 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.030066 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.030071 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.030688 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:33.030886 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.030891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.031553 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.031554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.032364 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.032367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.032705 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.032705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.032822 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.033279 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:33.033287 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:33.034655 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:33.036142 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.036150 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.036152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.036399 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.036591 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.036592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.036592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.036683 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.036916 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:33.036919 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:33.037032 (+ 113us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.037109 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.037226 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260927 07:11:33.038025 18838 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40505
I20260927 07:11:33.038111 18838 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:33.038280 18838 heartbeater.cc:511] Master 127.4.174.126:40505 requested a full tablet report, sending...
I20260927 07:11:33.038683 18513 ts_manager.cc:194] Registered new tserver with Master: fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
I20260927 07:11:33.039186 18513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:42733
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:11:33.061153 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40505
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:33.141727 18843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:33.141932 18843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:33.141952 18843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:33.143108 18843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:33.143149 18843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:33.143174 18843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:11:33.144577 18843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:33.144789 18843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:33.145000 18843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:33.149649 18851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:33.149693 18843 server_base.cc:1034] running on GCE node
W20260927 07:11:33.149842 18848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:33.149874 18849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:33.150095 18843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:33.150300 18843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:33.151460 18843 hybrid_clock.cc:648] HybridClock initialized: now 1790493093151446 us; error 27 us; skew 500 ppm
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:33.153662 18843 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:11:33.154057 18843 webserver.cc:460] Webserver started at http://127.4.174.67:43009/ using document root <none> and password file <none>
I20260927 07:11:33.154224 18843 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:33.154263 18843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:33.154349 18843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:33.155336 18843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "22cb69a17e744254b6a8988b90adddf0"
format_stamp: "Formatted at 2026-09-27 07:11:33 on dist-test-slave-zw7p"
server_key: "99ac2af4a1e81d1447234f0f70390d9f"
server_key_iv: "64b3ada0fa8dce5cfe04860726adad68"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.155627 18843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "22cb69a17e744254b6a8988b90adddf0"
format_stamp: "Formatted at 2026-09-27 07:11:33 on dist-test-slave-zw7p"
server_key: "99ac2af4a1e81d1447234f0f70390d9f"
server_key_iv: "64b3ada0fa8dce5cfe04860726adad68"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.156729 18843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 07:11:33.157301 18858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.157457 18843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:11:33.157546 18843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "22cb69a17e744254b6a8988b90adddf0"
format_stamp: "Formatted at 2026-09-27 07:11:33 on dist-test-slave-zw7p"
server_key: "99ac2af4a1e81d1447234f0f70390d9f"
server_key_iv: "64b3ada0fa8dce5cfe04860726adad68"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.157603 18843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:33.180064 18843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:33.181891 18843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:33.182030 18843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:33.182243 18843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:33.182590 18843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:33.182631 18843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.182662 18843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:33.182685 18843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.188162 18843 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:32935
I20260927 07:11:33.188211 18971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:32935 every 8 connection(s)
I20260927 07:11:33.188524 18843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:33.195997 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 18843
I20260927 07:11:33.196087 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:11:33.196265 4793 external_mini_cluster.cc:1428] Setting key b38600de8bc2373e6d0965255a1327b5
I20260927 07:11:33.197427 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.189216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:55251 (local address 127.4.174.126:40505)
0927 07:11:33.189266 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.189268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.189513 (+ 245us) server_negotiation.cc:407] Connection header received
0927 07:11:33.190020 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.190021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.190032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.190084 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:33.191059 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.191582 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.192543 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.192736 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.192790 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.194803 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:33.194804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:33.194804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:33.194813 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:33.196122 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.196509 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.196510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.196511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.196571 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.196824 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.196824 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.196825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.196960 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:33.197014 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.197298 (+ 284us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.197362 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:11:33.197618 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.189190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:55251)
0927 07:11:33.189381 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.189428 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.189925 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.190154 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.190159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.190753 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:33.190953 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.190958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.191665 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.191666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.192465 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.192467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.192801 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.192802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.192915 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.193364 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:33.193372 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:33.194717 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:33.196198 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.196204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.196206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.196454 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.196634 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.196635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.196635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.196765 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.197020 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:33.197026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:33.197184 (+ 158us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.197275 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.197411 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":76,"threads_started":1}
I20260927 07:11:33.198038 18972 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40505
I20260927 07:11:33.198122 18972 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:33.198288 18972 heartbeater.cc:511] Master 127.4.174.126:40505 requested a full tablet report, sending...
I20260927 07:11:33.198655 18513 ts_manager.cc:194] Registered new tserver with Master: 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935)
I20260927 07:11:33.198990 18513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:55251
I20260927 07:11:33.207921 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:11:33.214437 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.209207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.4.174.126:40505)
0927 07:11:33.209244 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.209246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.209346 (+ 100us) server_negotiation.cc:407] Connection header received
0927 07:11:33.209406 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.209406 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.209415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.209463 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:33.209980 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.210471 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.211185 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.211368 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.211415 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.212392 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:33.212393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:33.212393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:33.212399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:33.213583 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.213866 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.213867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.213867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.213916 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.214046 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.214047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.214047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.214173 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:33.214242 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.214316 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.214374 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:11:33.216149 18513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:11:33.217713 18513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:11:33.224000 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.219818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:32935 (local address 127.0.0.1:45278)
0927 07:11:33.219964 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.220003 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.220063 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.220464 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.220467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.220469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.220520 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.220521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.221792 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.221792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.222825 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.222827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.223702 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.223703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.223780 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.223793 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.223871 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.223926 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":36,"thread_start_us":67,"threads_started":1}
I20260927 07:11:33.224334 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.220030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39353 (local address 127.0.0.1:41850)
0927 07:11:33.220660 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.220668 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.220709 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.220940 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.220941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.220942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.220994 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.220995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.222165 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.222166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.223241 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.223243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.224098 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.224100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.224167 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.224171 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.224219 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.224258 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260927 07:11:33.224396 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.219951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.4.174.67:32935)
0927 07:11:33.220322 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.220326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.220348 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:11:33.220379 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.220381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.220393 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.220456 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.220593 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.221691 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.222895 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.223523 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.223609 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.223668 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.224278 (+ 610us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.224323 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":74,"threads_started":1}
I20260927 07:11:33.224617 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.220147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.4.174.66:39353)
0927 07:11:33.220311 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.220316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.220709 (+ 393us) server_negotiation.cc:407] Connection header received
0927 07:11:33.220779 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.220781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.220796 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.220860 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.221062 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.222067 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.223325 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.223936 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.224004 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.224062 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.224494 (+ 432us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.224540 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260927 07:11:33.225070 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.219822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45371 (local address 127.0.0.1:38814)
0927 07:11:33.220525 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.220533 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.220592 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.220962 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.220963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.220965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.221017 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.221018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.221834 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.221835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.223079 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.223082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.224017 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.224018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.224326 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.224331 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.224959 (+ 628us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.225005 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":29,"thread_start_us":89,"threads_started":1}
I20260927 07:11:33.225135 18906 tablet_service.cc:1467] Processing CreateTablet for tablet 77f54b32067346c1a3e9e7a074265dbd (DEFAULT_TABLE table=test-table [id=14d791ef0a944b0596270be155466e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:33.225332 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.220132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.4.174.65:45371)
0927 07:11:33.220311 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.220316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.220763 (+ 447us) server_negotiation.cc:407] Connection header received
0927 07:11:33.220804 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.220807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.220822 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.220890 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.221079 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.221746 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.223160 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.223878 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.223932 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.223986 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.225214 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.225261 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260927 07:11:33.225302 18772 tablet_service.cc:1467] Processing CreateTablet for tablet 77f54b32067346c1a3e9e7a074265dbd (DEFAULT_TABLE table=test-table [id=14d791ef0a944b0596270be155466e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:33.225498 18906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f54b32067346c1a3e9e7a074265dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:33.225633 18772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f54b32067346c1a3e9e7a074265dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:33.225916 18638 tablet_service.cc:1467] Processing CreateTablet for tablet 77f54b32067346c1a3e9e7a074265dbd (DEFAULT_TABLE table=test-table [id=14d791ef0a944b0596270be155466e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:33.226203 18638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f54b32067346c1a3e9e7a074265dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:33.228163 18989 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: Bootstrap starting.
I20260927 07:11:33.228715 18989 tablet_bootstrap.cc:654] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:33.228746 18990 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Bootstrap starting.
I20260927 07:11:33.229254 18989 log.cc:824] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:33.229254 18991 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Bootstrap starting.
I20260927 07:11:33.229362 18990 tablet_bootstrap.cc:654] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:33.229851 18991 tablet_bootstrap.cc:654] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:33.229866 18990 log.cc:824] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:33.229975 18989 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: No bootstrap required, opened a new log
I20260927 07:11:33.230046 18989 ts_tablet_manager.cc:1397] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:11:33.230388 18991 log.cc:824] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:33.230602 18990 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: No bootstrap required, opened a new log
I20260927 07:11:33.230656 18990 ts_tablet_manager.cc:1397] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:11:33.230985 18991 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: No bootstrap required, opened a new log
I20260927 07:11:33.231027 18991 ts_tablet_manager.cc:1397] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:11:33.231904 18989 raft_consensus.cc:348] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.232079 18989 raft_consensus.cc:374] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:33.232110 18989 raft_consensus.cc:729] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22cb69a17e744254b6a8988b90adddf0, State: Initialized, Role: FOLLOWER
I20260927 07:11:33.232227 18989 consensus_queue.cc:260] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.232307 18991 raft_consensus.cc:348] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.232456 18991 raft_consensus.cc:374] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:33.232491 18991 raft_consensus.cc:729] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eceb0933a873433794d22e355afa18b0, State: Initialized, Role: FOLLOWER
I20260927 07:11:33.232501 18989 ts_tablet_manager.cc:1428] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:11:33.232522 18972 heartbeater.cc:503] Master 127.4.174.126:40505 was elected leader, sending a full tablet report...
I20260927 07:11:33.232630 18991 consensus_queue.cc:260] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.232518 18990 raft_consensus.cc:348] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.232709 18990 raft_consensus.cc:374] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:33.232765 18990 raft_consensus.cc:729] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc562b57ab3f48008b96bbcb836e4486, State: Initialized, Role: FOLLOWER
I20260927 07:11:33.232892 18990 consensus_queue.cc:260] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.233016 18991 ts_tablet_manager.cc:1428] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:11:33.233144 18838 heartbeater.cc:503] Master 127.4.174.126:40505 was elected leader, sending a full tablet report...
I20260927 07:11:33.233264 18704 heartbeater.cc:503] Master 127.4.174.126:40505 was elected leader, sending a full tablet report...
I20260927 07:11:33.233145 18990 ts_tablet_manager.cc:1428] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260927 07:11:33.279050 18839 tablet.cc:2369] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:11:33.300153 18996 raft_consensus.cc:482] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:33.300283 18996 raft_consensus.cc:504] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.300588 18996 leader_election.cc:288] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935), fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
I20260927 07:11:33.303653 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.300963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:46923 (local address 127.4.174.66:39353)
0927 07:11:33.301008 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.301010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.301152 (+ 142us) server_negotiation.cc:407] Connection header received
0927 07:11:33.301218 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.301219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.301228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.301275 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.301430 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.301868 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.302928 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.303330 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.303402 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.303410 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.303557 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.303590 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:11:33.303653 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.300778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:32935 (local address 127.4.174.65:37323)
0927 07:11:33.300858 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.300872 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.300942 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.301117 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.301118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.301120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.301197 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.301199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.301936 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.301937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.302833 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.302836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.303399 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.303400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.303479 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.303483 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.303529 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.303570 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":44}
I20260927 07:11:33.303659 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.300886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:37323 (local address 127.4.174.67:32935)
0927 07:11:33.300940 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.300942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.300952 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:11:33.301026 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.301026 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.301037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.301104 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.301269 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.301857 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.302921 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.303322 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.303393 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.303403 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.303549 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.303585 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 07:11:33.303673 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.300890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39353 (local address 127.4.174.65:46923)
0927 07:11:33.301084 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.301098 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.301149 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.301291 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.301291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.301293 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.301367 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.301368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.301950 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.301951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.302846 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.302848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.303410 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.303411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.303474 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.303478 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.303532 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.303576 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":38,"thread_start_us":114,"threads_started":1}
I20260927 07:11:33.303978 18792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f54b32067346c1a3e9e7a074265dbd" candidate_uuid: "eceb0933a873433794d22e355afa18b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc562b57ab3f48008b96bbcb836e4486" is_pre_election: true
I20260927 07:11:33.304118 18792 raft_consensus.cc:2454] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eceb0933a873433794d22e355afa18b0 in term 0.
I20260927 07:11:33.304140 18926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f54b32067346c1a3e9e7a074265dbd" candidate_uuid: "eceb0933a873433794d22e355afa18b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22cb69a17e744254b6a8988b90adddf0" is_pre_election: true
I20260927 07:11:33.304302 18926 raft_consensus.cc:2454] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eceb0933a873433794d22e355afa18b0 in term 0.
I20260927 07:11:33.304338 18591 leader_election.cc:302] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eceb0933a873433794d22e355afa18b0, fc562b57ab3f48008b96bbcb836e4486; no voters:
I20260927 07:11:33.304526 18996 raft_consensus.cc:2781] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:11:33.304566 18996 raft_consensus.cc:482] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:11:33.304586 18996 raft_consensus.cc:3037] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:33.305236 18996 raft_consensus.cc:504] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.305352 18996 leader_election.cc:288] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 1 election: Requested vote from peers 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935), fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
I20260927 07:11:33.305526 18926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f54b32067346c1a3e9e7a074265dbd" candidate_uuid: "eceb0933a873433794d22e355afa18b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22cb69a17e744254b6a8988b90adddf0"
I20260927 07:11:33.305568 18792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f54b32067346c1a3e9e7a074265dbd" candidate_uuid: "eceb0933a873433794d22e355afa18b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc562b57ab3f48008b96bbcb836e4486"
I20260927 07:11:33.305598 18926 raft_consensus.cc:3037] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:33.305651 18792 raft_consensus.cc:3037] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:33.306265 18926 raft_consensus.cc:2454] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eceb0933a873433794d22e355afa18b0 in term 1.
I20260927 07:11:33.306428 18591 leader_election.cc:302] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22cb69a17e744254b6a8988b90adddf0, eceb0933a873433794d22e355afa18b0; no voters:
I20260927 07:11:33.306434 18792 raft_consensus.cc:2454] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eceb0933a873433794d22e355afa18b0 in term 1.
I20260927 07:11:33.306507 18996 raft_consensus.cc:2781] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:11:33.306690 18996 raft_consensus.cc:686] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 1 LEADER]: Becoming Leader. State: Replica: eceb0933a873433794d22e355afa18b0, State: Running, Role: LEADER
I20260927 07:11:33.306782 18996 consensus_queue.cc:237] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.307628 18513 catalog_manager.cc:5495] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to eceb0933a873433794d22e355afa18b0 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "eceb0933a873433794d22e355afa18b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } health_report { overall_health: UNKNOWN } } }
I20260927 07:11:33.317276 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.315170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38824 (local address 127.4.174.65:45371)
0927 07:11:33.315230 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.315231 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.315239 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:11:33.315280 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.315280 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.315297 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.315359 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:11:33.315466 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.315949 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.316629 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.316833 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.316885 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.316983 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:11:33.317084 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:11:33.317144 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.317157 (+ 13us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.317175 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 07:11:33.318953 18792 raft_consensus.cc:1261] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 1 FOLLOWER]: Refusing update from remote peer eceb0933a873433794d22e355afa18b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:33.319031 18926 raft_consensus.cc:1261] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 1 FOLLOWER]: Refusing update from remote peer eceb0933a873433794d22e355afa18b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:33.319259 18996 consensus_queue.cc:1035] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:33.319409 18999 consensus_queue.cc:1035] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:33.321837 19004 mvcc.cc:205] Tried to move back new op lower bound from 7333859710232023040 to 7333859710185304064. Current Snapshot: MvccSnapshot[applied={T|T < 7333859710232023040}]
I20260927 07:11:33.324265 18513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:11:33.324347 18513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:11:33.325033 18513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Sending DeleteTablet for 3 replicas of tablet 77f54b32067346c1a3e9e7a074265dbd
I20260927 07:11:33.325335 18906 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f54b32067346c1a3e9e7a074265dbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:33 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:45278
I20260927 07:11:33.325415 18638 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f54b32067346c1a3e9e7a074265dbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:33 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:38814
I20260927 07:11:33.325424 18772 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f54b32067346c1a3e9e7a074265dbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:33 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:41850
I20260927 07:11:33.325650 19012 tablet_replica.cc:330] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: stopping tablet replica
I20260927 07:11:33.325661 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 18575
I20260927 07:11:33.326273 19012 raft_consensus.cc:2229] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:11:33.326314 19013 tablet_replica.cc:330] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: stopping tablet replica
I20260927 07:11:33.326417 19012 raft_consensus.cc:2258] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:33.326450 19013 raft_consensus.cc:2229] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:11:33.326576 19013 raft_consensus.cc:2258] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:33.326876 19012 ts_tablet_manager.cc:1905] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:11:33.327258 19013 ts_tablet_manager.cc:1905] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:11:33.328509 19012 ts_tablet_manager.cc:1918] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:11:33.328580 19012 log.cc:1196] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/77f54b32067346c1a3e9e7a074265dbd
I20260927 07:11:33.328760 19013 ts_tablet_manager.cc:1918] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:11:33.328831 19013 log.cc:1196] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/77f54b32067346c1a3e9e7a074265dbd
I20260927 07:11:33.328872 19012 ts_tablet_manager.cc:1939] T 77f54b32067346c1a3e9e7a074265dbd P 22cb69a17e744254b6a8988b90adddf0: Deleting consensus metadata
I20260927 07:11:33.329102 19013 ts_tablet_manager.cc:1939] T 77f54b32067346c1a3e9e7a074265dbd P fc562b57ab3f48008b96bbcb836e4486: Deleting consensus metadata
I20260927 07:11:33.329284 18497 catalog_manager.cc:4841] TS 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935): tablet 77f54b32067346c1a3e9e7a074265dbd (table test-table [id=14d791ef0a944b0596270be155466e79]) successfully deleted
I20260927 07:11:33.329469 18497 catalog_manager.cc:4841] TS fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353): tablet 77f54b32067346c1a3e9e7a074265dbd (table test-table [id=14d791ef0a944b0596270be155466e79]) successfully deleted
W20260927 07:11:33.330801 18498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:45371 (error 108)
I20260927 07:11:33.331029 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 18709
I20260927 07:11:33.331565 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.331149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45371 (local address 127.0.0.1:38826)
0927 07:11:33.331505 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.331538 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:45371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339}
W20260927 07:11:33.331910 18498 catalog_manager.cc:4584] TS eceb0933a873433794d22e355afa18b0 (127.4.174.65:45371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77f54b32067346c1a3e9e7a074265dbd: Network error: Client connection negotiation failed: client connection to 127.4.174.65:45371: connect: Connection refused (error 111)
I20260927 07:11:33.335089 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 18843
I20260927 07:11:33.339493 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 18481
I20260927 07:11:33.343688 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40505
--webserver_interface=127.4.174.126
--webserver_port=40655
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:33.423954 19016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:33.424154 19016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:33.424183 19016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:33.425307 19016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:33.425351 19016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:33.425369 19016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:33.425384 19016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:33.425398 19016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:33.427134 19016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=40655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:33.427378 19016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:33.427579 19016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:33.435448 19023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:33.435431 19025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:33.435417 19022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:33.435779 19016 server_base.cc:1034] running on GCE node
I20260927 07:11:33.436029 19016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:33.436321 19016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:33.437457 19016 hybrid_clock.cc:648] HybridClock initialized: now 1790493093437440 us; error 32 us; skew 500 ppm
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:33.439857 19016 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:11:33.440292 19016 webserver.cc:460] Webserver started at http://127.4.174.126:40655/ using document root <none> and password file <none>
I20260927 07:11:33.440490 19016 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:33.440538 19016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:33.441805 19016 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:33.442415 19032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.442611 19016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:33.442700 19016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "7c172b93c2bc4db8bb48da90bcf3eecf"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "51dbfd0a7d0f100fcf2b307b0c0a1a19"
server_key_iv: "f2ba00e42ff143f20e12ea8b15d9e2d7"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.443095 19016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:33.453464 19016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:33.455353 19016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:33.455487 19016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:33.459312 19016 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40505
I20260927 07:11:33.459349 19084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40505 every 8 connection(s)
I20260927 07:11:33.459708 19016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:11:33.461308 19085 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:11:33.461979 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Bootstrap starting.
I20260927 07:11:33.468015 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 19016
I20260927 07:11:33.468253 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:45371
--local_ip_for_outbound_sockets=127.4.174.65
--tserver_master_addrs=127.4.174.126:40505
--webserver_port=42347
--webserver_interface=127.4.174.65
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:11:33.469882 19085 log.cc:824] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:33.471735 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:11:33.471994 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Bootstrap complete.
I20260927 07:11:33.474052 19085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:33.474272 19085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c172b93c2bc4db8bb48da90bcf3eecf, State: Initialized, Role: FOLLOWER
I20260927 07:11:33.474421 19085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:33.474488 19085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:11:33.474534 19085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:11:33.474598 19085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:11:33.475600 19085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:33.475688 19085 leader_election.cc:302] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c172b93c2bc4db8bb48da90bcf3eecf; no voters:
I20260927 07:11:33.475847 19085 leader_election.cc:288] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 07:11:33.475905 19089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 2 FOLLOWER]: Leader election won for term 2
I20260927 07:11:33.476016 19089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 2 LEADER]: Becoming Leader. State: Replica: 7c172b93c2bc4db8bb48da90bcf3eecf, State: Running, Role: LEADER
I20260927 07:11:33.476105 19085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:33.476171 19089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:33.476531 19089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c172b93c2bc4db8bb48da90bcf3eecf. Latest consensus state: current_term: 2 leader_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } } }
I20260927 07:11:33.476706 19091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } } }
I20260927 07:11:33.476969 19091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:33.477123 19089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:33.477248 19099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:11:33.478024 19099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:11:33.478116 19099 catalog_manager.cc:1259] Loaded cluster ID: 3d1ee5314a0c43949f829d433e23e04a
I20260927 07:11:33.478148 19099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:11:33.478343 19099 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260927 07:11:33.481041 19099 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260927 07:11:33.482873 19099 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:11:33.483052 19099 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Loaded TSK: 0
I20260927 07:11:33.483261 19099 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 07:11:33.554500 19087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:33.554734 19087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:33.554769 19087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:33.555966 19087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:33.556007 19087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:33.556041 19087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:11:33.557890 19087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:45371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=42347
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:33.558135 19087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:33.558367 19087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:33.563262 19115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:33.563301 19112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:33.563293 19113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:33.563681 19087 server_base.cc:1034] running on GCE node
I20260927 07:11:33.563879 19087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:33.564108 19087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:33.565274 19087 hybrid_clock.cc:648] HybridClock initialized: now 1790493093565257 us; error 32 us; skew 500 ppm
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:33.571543 19087 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:11:33.572064 19087 webserver.cc:460] Webserver started at http://127.4.174.65:42347/ using document root <none> and password file <none>
I20260927 07:11:33.572294 19087 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:33.572361 19087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:33.573745 19087 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:33.574355 19122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.574512 19087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:33.574615 19087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "eceb0933a873433794d22e355afa18b0"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "3aa68033ebdf92fc77dcbde9325b9b46"
server_key_iv: "ad54dc48a4ae81c3423ed9f4cb4601f5"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.574970 19087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:33.590155 19087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:33.592316 19087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:33.592428 19087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:33.592644 19087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:33.593104 19129 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 07:11:33.595045 19087 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 07:11:33.595101 19087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 07:11:33.595135 19087 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 07:11:33.595767 19087 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 07:11:33.595830 19087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 07:11:33.595883 19129 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Bootstrap starting.
I20260927 07:11:33.602278 19087 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:45371
I20260927 07:11:33.602402 19236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:45371 every 8 connection(s)
I20260927 07:11:33.602741 19087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:11:33.603240 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 19087
I20260927 07:11:33.603498 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:39353
--local_ip_for_outbound_sockets=127.4.174.66
--tserver_master_addrs=127.4.174.126:40505
--webserver_port=39659
--webserver_interface=127.4.174.66
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:11:33.606647 19129 log.cc:824] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:33.609386 19129 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:11:33.609701 19129 tablet_bootstrap.cc:492] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Bootstrap complete.
I20260927 07:11:33.610863 19129 ts_tablet_manager.cc:1397] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260927 07:11:33.612249 19129 raft_consensus.cc:348] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.612587 19129 raft_consensus.cc:729] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eceb0933a873433794d22e355afa18b0, State: Initialized, Role: FOLLOWER
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:33.612836 19129 consensus_queue.cc:260] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } }
I20260927 07:11:33.613881 19129 ts_tablet_manager.cc:1428] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260927 07:11:33.617944 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.605110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:53347 (local address 127.4.174.126:40505)
0927 07:11:33.605361 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.605371 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.606253 (+ 882us) server_negotiation.cc:407] Connection header received
0927 07:11:33.606402 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.606409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.606462 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.606625 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:33.607548 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.608919 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.610472 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.611140 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.611247 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.614236 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:33.614241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:33.614241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:33.614255 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:33.616213 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.616703 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.616706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.616706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.616812 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.617060 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.617061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.617061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.617273 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:33.617336 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.617656 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.617801 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260927 07:11:33.617971 19240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.605063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:53347)
0927 07:11:33.605460 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.605505 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.606094 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.606766 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.606770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.607114 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:33.607405 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.607413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.609053 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.609055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.610371 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.610375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.611381 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.611383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.611538 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.612157 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:33.612167 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:33.614117 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:33.616316 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.616323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.616325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.616619 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.616880 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.616881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.616881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.617004 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.617330 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:33.617334 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:33.617500 (+ 166us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.617608 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.617773 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":90,"threads_started":1}
I20260927 07:11:33.618662 19237 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40505
I20260927 07:11:33.618760 19237 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:33.618933 19237 heartbeater.cc:511] Master 127.4.174.126:40505 requested a full tablet report, sending...
I20260927 07:11:33.619650 19049 ts_manager.cc:194] Registered new tserver with Master: eceb0933a873433794d22e355afa18b0 (127.4.174.65:45371)
I20260927 07:11:33.620643 19049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:53347
I20260927 07:11:33.621548 19237 heartbeater.cc:503] Master 127.4.174.126:40505 was elected leader, sending a full tablet report...
I20260927 07:11:33.625978 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.620216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45371 (local address 127.0.0.1:38828)
0927 07:11:33.620450 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.620487 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.620548 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.622805 (+ 2257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.622807 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.622809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.622862 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.622863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.624041 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.624041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.624829 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.624832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.625717 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.625719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.625786 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.625797 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.625883 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.625924 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
I20260927 07:11:33.625984 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.620277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.4.174.65:45371)
0927 07:11:33.622644 (+ 2367us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.622648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.622657 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:11:33.622686 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.622688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.622703 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.622780 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.622924 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.623975 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.624909 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.625625 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.625697 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.625749 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.625883 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.625921 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2337,"thread_start_us":88,"threads_started":1}
I20260927 07:11:33.626348 19171 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f54b32067346c1a3e9e7a074265dbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:33 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:38828
I20260927 07:11:33.626856 19246 tablet_replica.cc:330] stopping tablet replica
I20260927 07:11:33.626960 19246 raft_consensus.cc:2229] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:11:33.627022 19246 raft_consensus.cc:2258] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:33.627326 19246 ts_tablet_manager.cc:1905] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:11:33.629518 19246 ts_tablet_manager.cc:1918] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:11:33.629588 19246 log.cc:1196] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/77f54b32067346c1a3e9e7a074265dbd
I20260927 07:11:33.629786 19246 ts_tablet_manager.cc:1939] T 77f54b32067346c1a3e9e7a074265dbd P eceb0933a873433794d22e355afa18b0: Deleting consensus metadata
I20260927 07:11:33.630388 19034 catalog_manager.cc:4841] TS eceb0933a873433794d22e355afa18b0 (127.4.174.65:45371): tablet 77f54b32067346c1a3e9e7a074265dbd (table test-table [id=14d791ef0a944b0596270be155466e79]) successfully deleted
W20260927 07:11:33.689301 19239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:33.689502 19239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:33.689538 19239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:33.690697 19239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:33.690734 19239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:33.690768 19239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:11:33.692224 19239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:39353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=39659
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:33.692443 19239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:33.692653 19239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:33.697369 19251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:33.697546 19239 server_base.cc:1034] running on GCE node
W20260927 07:11:33.697515 19254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:33.697425 19252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:33.697820 19239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:33.698060 19239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:33.699203 19239 hybrid_clock.cc:648] HybridClock initialized: now 1790493093699186 us; error 30 us; skew 500 ppm
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:33.701514 19239 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:11:33.701946 19239 webserver.cc:460] Webserver started at http://127.4.174.66:39659/ using document root <none> and password file <none>
I20260927 07:11:33.702142 19239 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:33.702201 19239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:33.703581 19239 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:33.704120 19261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.704272 19239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:33.704362 19239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "fc562b57ab3f48008b96bbcb836e4486"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "9c865944a0c3bc87e2dd2d1c515a62fa"
server_key_iv: "26cffcc43b4848d54698b80a0fa277f4"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.704698 19239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:33.717720 19239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:33.719818 19239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:33.719962 19239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:33.720165 19239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:33.720440 19239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:33.720472 19239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.720500 19239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:33.720521 19239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.725795 19239 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:39353
I20260927 07:11:33.725852 19374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:39353 every 8 connection(s)
I20260927 07:11:33.726157 19239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:11:33.728905 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 19239
I20260927 07:11:33.729174 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:32935
--local_ip_for_outbound_sockets=127.4.174.67
--tserver_master_addrs=127.4.174.126:40505
--webserver_port=43009
--webserver_interface=127.4.174.67
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:33.735407 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.726786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:54931 (local address 127.4.174.126:40505)
0927 07:11:33.726879 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.726881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.727044 (+ 163us) server_negotiation.cc:407] Connection header received
0927 07:11:33.727514 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.727515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.727524 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.727572 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:33.728410 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.729018 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.730057 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.730341 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.730402 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.732795 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:33.732797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:33.732797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:33.732804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:33.734027 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.734488 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.734489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.734489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.734534 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.734802 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.734802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.734803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.734905 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:33.734951 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.735266 (+ 315us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.735342 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:11:33.735662 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.726780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:54931)
0927 07:11:33.726933 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.726970 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.727438 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.727626 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.727629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.728152 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:33.728344 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.728349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.729103 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.729104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.729974 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.729976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.730430 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.730431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.730594 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.731066 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:33.731076 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:33.732702 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:33.734106 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.734114 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.734116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.734418 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.734604 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.734604 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.734605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.734742 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.734962 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:33.734968 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:33.735136 (+ 168us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.735253 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.735429 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260927 07:11:33.736142 19375 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40505
I20260927 07:11:33.736233 19375 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:33.736402 19375 heartbeater.cc:511] Master 127.4.174.126:40505 requested a full tablet report, sending...
I20260927 07:11:33.736768 19049 ts_manager.cc:194] Registered new tserver with Master: fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
I20260927 07:11:33.737247 19049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:54931
W20260927 07:11:33.809314 19378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:33.809515 19378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:33.809542 19378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:33.810705 19378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:33.810743 19378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:33.810768 19378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:11:33.812162 19378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:32935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=43009
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:33.812343 19378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:33.812521 19378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:33.817233 19386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:33.817287 19384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:33.817342 19378 server_base.cc:1034] running on GCE node
W20260927 07:11:33.817233 19383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:33.817763 19378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:33.817996 19378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:33.819135 19378 hybrid_clock.cc:648] HybridClock initialized: now 1790493093819124 us; error 30 us; skew 500 ppm
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:33.821405 19378 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:11:33.821807 19378 webserver.cc:460] Webserver started at http://127.4.174.67:43009/ using document root <none> and password file <none>
I20260927 07:11:33.821997 19378 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:33.822057 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:33.823369 19378 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:33.823895 19393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.824040 19378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:33.824115 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "22cb69a17e744254b6a8988b90adddf0"
format_stamp: "Formatted at 2026-09-27 07:11:33 on dist-test-slave-zw7p"
server_key: "99ac2af4a1e81d1447234f0f70390d9f"
server_key_iv: "64b3ada0fa8dce5cfe04860726adad68"
server_key_version: "encryptionkey@0"
I20260927 07:11:33.824434 19378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:33.849341 19378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:33.851445 19378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:33.851581 19378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:33.851792 19378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:33.852097 19378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:33.852129 19378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.852159 19378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:33.852183 19378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.857599 19378 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:32935
I20260927 07:11:33.857657 19506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:32935 every 8 connection(s)
I20260927 07:11:33.857969 19378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:11:33 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493093, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:33.864352 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 19378
I20260927 07:11:33.866837 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.858622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:39783 (local address 127.4.174.126:40505)
0927 07:11:33.858669 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.858671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.858897 (+ 226us) server_negotiation.cc:407] Connection header received
0927 07:11:33.859388 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.859389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.859406 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.859457 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:33.860369 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.860852 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.861799 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.861988 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.862043 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.864037 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:33.864038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:33.864039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:33.864047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:33.865499 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.865944 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.865945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.865945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.865995 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.866231 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.866232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.866232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.866339 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:33.866397 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.866664 (+ 267us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.866742 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:11:33.866942 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.858616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:39783)
0927 07:11:33.858773 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.858811 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.859285 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.859532 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.859536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.860084 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:33.860273 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.860278 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.860930 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.860931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.861719 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.861721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.862056 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.862057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.862176 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.862634 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:33.862642 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:33.863951 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:33.865578 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.865584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.865586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.865875 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.866048 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:33.866049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:33.866049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:33.866178 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:33.866398 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:33.866403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:33.866519 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.866639 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.866762 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260927 07:11:33.867400 19507 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40505
I20260927 07:11:33.867501 19507 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:33.867689 19507 heartbeater.cc:511] Master 127.4.174.126:40505 requested a full tablet report, sending...
I20260927 07:11:33.868072 19049 ts_manager.cc:194] Registered new tserver with Master: 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935)
I20260927 07:11:33.868566 19049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:39783
I20260927 07:11:33.870726 19511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.864806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.4.174.126:40505)
0927 07:11:33.864931 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.864933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.864942 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:11:33.864961 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.864962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.864971 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.865023 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:33.865646 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.866314 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.867310 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.867572 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.867643 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.868791 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:33.868792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:33.868792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:33.868798 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:33.869944 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.870208 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.870208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.870209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.870251 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:33.870358 (+ 107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:33.870359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:33.870359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:33.870441 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:33.870542 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.870598 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.870666 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260927 07:11:33.871258 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:11:33.873451 19511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.871528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33306 (local address 127.4.174.126:40505)
0927 07:11:33.871561 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.871563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.871661 (+ 98us) server_negotiation.cc:407] Connection header received
0927 07:11:33.871706 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.871707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.871733 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.871782 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:11:33.871856 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.872274 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.872920 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.873087 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.873125 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.873225 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:11:33.873307 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:11:33.873370 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.873382 (+ 12us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.873396 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:11:33.873679 19049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:11:33.875247 19049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:11:33.877264 19171 tablet_service.cc:1467] Processing CreateTablet for tablet 102fabfd64cc4481951f6d706c06d930 (DEFAULT_TABLE table=test-table [id=5dc5f6f8ce804bff90c559b8c4a27b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:33.877476 19171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102fabfd64cc4481951f6d706c06d930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:33.878683 19129 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Bootstrap starting.
I20260927 07:11:33.879033 19129 tablet_bootstrap.cc:654] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:33.879585 19129 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: No bootstrap required, opened a new log
I20260927 07:11:33.879637 19129 ts_tablet_manager.cc:1397] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260927 07:11:33.879743 19129 raft_consensus.cc:348] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.879817 19129 raft_consensus.cc:374] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:33.879855 19129 raft_consensus.cc:729] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eceb0933a873433794d22e355afa18b0, State: Initialized, Role: FOLLOWER
I20260927 07:11:33.879894 19129 consensus_queue.cc:260] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.879971 19129 ts_tablet_manager.cc:1428] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:33.881912 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.876957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50072 (local address 127.4.174.67:32935)
0927 07:11:33.878739 (+ 1782us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.878744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.878754 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:11:33.878788 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.878789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.878805 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.878873 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.878996 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.880090 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.881131 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.881602 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.881653 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.881707 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.881801 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.881842 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1749,"thread_start_us":86,"threads_started":1}
I20260927 07:11:33.881908 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.876896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:32935 (local address 127.0.0.1:50072)
0927 07:11:33.877014 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.877024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.877065 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.878883 (+ 1818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.878884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.878886 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.878921 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.878922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.880195 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.880196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.881056 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.881059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.881673 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.881674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.881729 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.881731 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.881775 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.881816 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260927 07:11:33.881974 19514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.876979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39353 (local address 127.0.0.1:50142)
0927 07:11:33.877107 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.877116 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.877159 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.879131 (+ 1972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.879131 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.879133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.879165 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.879167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.880242 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.880243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.881058 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.881061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.881699 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.881700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.881774 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.881778 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.881827 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.881863 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260927 07:11:33.881947 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.877077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.4.174.66:39353)
0927 07:11:33.879006 (+ 1929us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.879010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.879020 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:11:33.879053 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.879054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.879066 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.879130 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.879211 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.880130 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.881132 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.881616 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.881680 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.881737 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.881838 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.881878 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1886,"thread_start_us":75,"threads_started":1}
I20260927 07:11:33.882731 19441 tablet_service.cc:1467] Processing CreateTablet for tablet 102fabfd64cc4481951f6d706c06d930 (DEFAULT_TABLE table=test-table [id=5dc5f6f8ce804bff90c559b8c4a27b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:33.882766 19309 tablet_service.cc:1467] Processing CreateTablet for tablet 102fabfd64cc4481951f6d706c06d930 (DEFAULT_TABLE table=test-table [id=5dc5f6f8ce804bff90c559b8c4a27b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:33.883026 19441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102fabfd64cc4481951f6d706c06d930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:33.883038 19309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102fabfd64cc4481951f6d706c06d930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:33.885557 19517 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: Bootstrap starting.
I20260927 07:11:33.885581 19518 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Bootstrap starting.
I20260927 07:11:33.886142 19518 tablet_bootstrap.cc:654] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:33.886143 19517 tablet_bootstrap.cc:654] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:33.886576 19518 log.cc:824] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:33.886607 19517 log.cc:824] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:33.887204 19518 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: No bootstrap required, opened a new log
I20260927 07:11:33.887207 19517 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: No bootstrap required, opened a new log
I20260927 07:11:33.887256 19517 ts_tablet_manager.cc:1397] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 07:11:33.887269 19518 ts_tablet_manager.cc:1397] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:11:33.888525 19518 raft_consensus.cc:348] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.888525 19517 raft_consensus.cc:348] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.888688 19517 raft_consensus.cc:374] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:33.888687 19518 raft_consensus.cc:374] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:33.888733 19518 raft_consensus.cc:729] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc562b57ab3f48008b96bbcb836e4486, State: Initialized, Role: FOLLOWER
I20260927 07:11:33.888761 19517 raft_consensus.cc:729] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22cb69a17e744254b6a8988b90adddf0, State: Initialized, Role: FOLLOWER
I20260927 07:11:33.888880 19517 consensus_queue.cc:260] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.888949 19518 consensus_queue.cc:260] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.889154 19517 ts_tablet_manager.cc:1428] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:11:33.889211 19518 ts_tablet_manager.cc:1428] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:11:33.889278 19375 heartbeater.cc:503] Master 127.4.174.126:40505 was elected leader, sending a full tablet report...
I20260927 07:11:33.889151 19507 heartbeater.cc:503] Master 127.4.174.126:40505 was elected leader, sending a full tablet report...
I20260927 07:11:33.973519 19243 raft_consensus.cc:482] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:33.973645 19243 raft_consensus.cc:504] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.973971 19243 leader_election.cc:288] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935), fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
W20260927 07:11:33.976644 19376 tablet.cc:2369] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:11:33.977603 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.974334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:53893 (local address 127.4.174.66:39353)
0927 07:11:33.974375 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.974376 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.974459 (+ 83us) server_negotiation.cc:407] Connection header received
0927 07:11:33.974503 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.974504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.974511 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.974578 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.974681 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.975385 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.976811 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.977221 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.977280 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.977288 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.977475 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.977543 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:11:33.977610 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.974244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39353 (local address 127.4.174.65:53893)
0927 07:11:33.974398 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.974406 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.974456 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.974572 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.974573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.974574 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.974634 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.974635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.975451 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.975452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.976699 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.976702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.977309 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.977311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.977397 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.977402 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.977448 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.977505 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":38,"thread_start_us":94,"threads_started":1}
I20260927 07:11:33.977720 19240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.974141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:32935 (local address 127.4.174.65:34199)
0927 07:11:33.974192 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:33.974202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:33.974245 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:33.974438 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:33.974439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:33.974441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:33.974512 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.974514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.975435 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.975435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.976699 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:33.976702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.977468 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.977469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:33.977578 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.977581 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:11:33.977625 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 07:11:33.977663 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":6}
I20260927 07:11:33.977887 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.974223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:34199 (local address 127.4.174.67:32935)
0927 07:11:33.974291 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.974293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.974302 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:11:33.974331 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.974332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.974341 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.974425 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:33.974605 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.975348 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.976799 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.977319 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.977428 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.977437 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.977779 (+ 342us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.977829 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 07:11:33.978143 19461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102fabfd64cc4481951f6d706c06d930" candidate_uuid: "eceb0933a873433794d22e355afa18b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22cb69a17e744254b6a8988b90adddf0" is_pre_election: true
I20260927 07:11:33.978154 19329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102fabfd64cc4481951f6d706c06d930" candidate_uuid: "eceb0933a873433794d22e355afa18b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc562b57ab3f48008b96bbcb836e4486" is_pre_election: true
I20260927 07:11:33.978268 19329 raft_consensus.cc:2454] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eceb0933a873433794d22e355afa18b0 in term 0.
I20260927 07:11:33.978281 19461 raft_consensus.cc:2454] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eceb0933a873433794d22e355afa18b0 in term 0.
I20260927 07:11:33.978502 19123 leader_election.cc:302] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eceb0933a873433794d22e355afa18b0, fc562b57ab3f48008b96bbcb836e4486; no voters:
I20260927 07:11:33.978657 19243 raft_consensus.cc:2781] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:11:33.978732 19243 raft_consensus.cc:482] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:11:33.978758 19243 raft_consensus.cc:3037] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:33.979447 19243 raft_consensus.cc:504] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.979570 19243 leader_election.cc:288] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 1 election: Requested vote from peers 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935), fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
I20260927 07:11:33.979750 19329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102fabfd64cc4481951f6d706c06d930" candidate_uuid: "eceb0933a873433794d22e355afa18b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc562b57ab3f48008b96bbcb836e4486"
I20260927 07:11:33.979760 19461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102fabfd64cc4481951f6d706c06d930" candidate_uuid: "eceb0933a873433794d22e355afa18b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22cb69a17e744254b6a8988b90adddf0"
I20260927 07:11:33.979816 19329 raft_consensus.cc:3037] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:33.979825 19461 raft_consensus.cc:3037] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:33.980468 19329 raft_consensus.cc:2454] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eceb0933a873433794d22e355afa18b0 in term 1.
I20260927 07:11:33.980532 19461 raft_consensus.cc:2454] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eceb0933a873433794d22e355afa18b0 in term 1.
I20260927 07:11:33.980630 19123 leader_election.cc:302] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eceb0933a873433794d22e355afa18b0, fc562b57ab3f48008b96bbcb836e4486; no voters:
I20260927 07:11:33.980731 19243 raft_consensus.cc:2781] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:11:33.980880 19243 raft_consensus.cc:686] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 LEADER]: Becoming Leader. State: Replica: eceb0933a873433794d22e355afa18b0, State: Running, Role: LEADER
I20260927 07:11:33.980943 19243 consensus_queue.cc:237] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:33.981649 19048 catalog_manager.cc:5495] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to eceb0933a873433794d22e355afa18b0 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "eceb0933a873433794d22e355afa18b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } health_report { overall_health: HEALTHY } } }
I20260927 07:11:33.995862 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:33.993784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.4.174.65:45371)
0927 07:11:33.993838 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:33.993839 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:33.993847 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:11:33.993904 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:33.993905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:33.993922 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:33.993978 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:11:33.994059 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.994526 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.995249 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:33.995444 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:33.995489 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:33.995593 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:11:33.995702 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:11:33.995751 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:33.995766 (+ 15us) server_negotiation.cc:299] Negotiation successful
0927 07:11:33.995784 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 07:11:33.997293 19461 raft_consensus.cc:1261] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 1 FOLLOWER]: Refusing update from remote peer eceb0933a873433794d22e355afa18b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:33.997315 19329 raft_consensus.cc:1261] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 1 FOLLOWER]: Refusing update from remote peer eceb0933a873433794d22e355afa18b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:33.997633 19524 consensus_queue.cc:1035] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:33.997752 19243 consensus_queue.cc:1035] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:33.999996 19526 mvcc.cc:205] Tried to move back new op lower bound from 7333859713010622464 to 7333859712946454528. Current Snapshot: MvccSnapshot[applied={T|T < 7333859713010622464}]
I20260927 07:11:34.000298 19527 mvcc.cc:205] Tried to move back new op lower bound from 7333859713010622464 to 7333859712946454528. Current Snapshot: MvccSnapshot[applied={T|T < 7333859713010622464}]
I20260927 07:11:34.002357 19049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:11:34.002437 19049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:11:34.003126 19049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Sending DeleteTablet for 3 replicas of tablet 102fabfd64cc4481951f6d706c06d930
I20260927 07:11:34.003386 19171 tablet_service.cc:1514] Processing DeleteTablet for tablet 102fabfd64cc4481951f6d706c06d930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:33 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:38828
I20260927 07:11:34.003465 19246 tablet_replica.cc:330] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: stopping tablet replica
I20260927 07:11:34.003504 19246 raft_consensus.cc:2229] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:11:34.003497 19309 tablet_service.cc:1514] Processing DeleteTablet for tablet 102fabfd64cc4481951f6d706c06d930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:33 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50142
I20260927 07:11:34.003501 19441 tablet_service.cc:1514] Processing DeleteTablet for tablet 102fabfd64cc4481951f6d706c06d930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:33 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:50072
I20260927 07:11:34.003578 19246 raft_consensus.cc:2258] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:34.003714 19535 tablet_replica.cc:330] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: stopping tablet replica
I20260927 07:11:34.003829 19535 raft_consensus.cc:2229] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:11:34.003881 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 19087
I20260927 07:11:34.003971 19535 raft_consensus.cc:2258] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:34.004063 19246 ts_tablet_manager.cc:1905] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:11:34.004374 19535 ts_tablet_manager.cc:1905] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:11:34.005895 19536 tablet_replica.cc:330] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: stopping tablet replica
I20260927 07:11:34.006023 19536 raft_consensus.cc:2229] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:11:34.006060 19535 ts_tablet_manager.cc:1918] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:11:34.006110 19535 log.cc:1196] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/102fabfd64cc4481951f6d706c06d930
I20260927 07:11:34.006116 19536 raft_consensus.cc:2258] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:34.006307 19535 ts_tablet_manager.cc:1939] T 102fabfd64cc4481951f6d706c06d930 P 22cb69a17e744254b6a8988b90adddf0: Deleting consensus metadata
I20260927 07:11:34.006536 19536 ts_tablet_manager.cc:1905] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:11:34.006717 19033 catalog_manager.cc:4841] TS 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935): tablet 102fabfd64cc4481951f6d706c06d930 (table test-table [id=5dc5f6f8ce804bff90c559b8c4a27b46]) successfully deleted
I20260927 07:11:34.007939 19536 ts_tablet_manager.cc:1918] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:11:34.007998 19536 log.cc:1196] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/102fabfd64cc4481951f6d706c06d930
I20260927 07:11:34.008212 19536 ts_tablet_manager.cc:1939] T 102fabfd64cc4481951f6d706c06d930 P fc562b57ab3f48008b96bbcb836e4486: Deleting consensus metadata
I20260927 07:11:34.008687 19033 catalog_manager.cc:4841] TS fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353): tablet 102fabfd64cc4481951f6d706c06d930 (table test-table [id=5dc5f6f8ce804bff90c559b8c4a27b46]) successfully deleted
W20260927 07:11:34.010141 19034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:45371 (error 108)
I20260927 07:11:34.010660 19514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.010508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45371 (local address 127.0.0.1:53110)
0927 07:11:34.010583 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.010625 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:45371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 07:11:34.010702 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 19239
W20260927 07:11:34.010820 19034 catalog_manager.cc:4584] TS eceb0933a873433794d22e355afa18b0 (127.4.174.65:45371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 102fabfd64cc4481951f6d706c06d930: Network error: Client connection negotiation failed: client connection to 127.4.174.65:45371: connect: Connection refused (error 111)
I20260927 07:11:34.015396 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 19378
I20260927 07:11:34.019277 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 19016
I20260927 07:11:34.023644 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40505
--webserver_interface=127.4.174.126
--webserver_port=40655
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:34.102480 19538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:34.102725 19538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:34.102748 19538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:34.103888 19538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:34.103922 19538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:34.103935 19538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:34.103946 19538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:34.103957 19538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:34.105345 19538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=40655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:34.105535 19538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:34.105700 19538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:34.110091 19543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:34.110193 19544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:34.110136 19546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:34.110357 19538 server_base.cc:1034] running on GCE node
I20260927 07:11:34.110597 19538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:34.110868 19538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:34.112004 19538 hybrid_clock.cc:648] HybridClock initialized: now 1790493094111993 us; error 28 us; skew 500 ppm
Sep 27 07:11:34 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493094, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:34.114180 19538 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:11:34.114576 19538 webserver.cc:460] Webserver started at http://127.4.174.126:40655/ using document root <none> and password file <none>
I20260927 07:11:34.114780 19538 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:34.114837 19538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:34.116060 19538 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:34.116596 19553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.116767 19538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:34.116873 19538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "7c172b93c2bc4db8bb48da90bcf3eecf"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "51dbfd0a7d0f100fcf2b307b0c0a1a19"
server_key_iv: "f2ba00e42ff143f20e12ea8b15d9e2d7"
server_key_version: "encryptionkey@0"
I20260927 07:11:34.117197 19538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:34.138482 19538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:34.140244 19538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:34.140378 19538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:34.144093 19538 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40505
I20260927 07:11:34.144130 19605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40505 every 8 connection(s)
I20260927 07:11:34.144498 19538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:11:34.145998 19606 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:11:34.146673 19606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Bootstrap starting.
I20260927 07:11:34.148051 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 19538
I20260927 07:11:34.148294 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:45371
--local_ip_for_outbound_sockets=127.4.174.65
--tserver_master_addrs=127.4.174.126:40505
--webserver_port=42347
--webserver_interface=127.4.174.65
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:11:34.154932 19606 log.cc:824] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:34.157020 19606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:11:34.157270 19606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf: Bootstrap complete.
I20260927 07:11:34.159314 19606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:34.159562 19606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c172b93c2bc4db8bb48da90bcf3eecf, State: Initialized, Role: FOLLOWER
I20260927 07:11:34.159713 19606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:34.159766 19606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:11:34.159811 19606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:11:34.159853 19606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 2 FOLLOWER]: Advancing to term 3
I20260927 07:11:34.160842 19606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:34.160938 19606 leader_election.cc:302] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c172b93c2bc4db8bb48da90bcf3eecf; no voters:
I20260927 07:11:34.161087 19606 leader_election.cc:288] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 07:11:34.161142 19610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 3 FOLLOWER]: Leader election won for term 3
I20260927 07:11:34.161252 19610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [term 3 LEADER]: Becoming Leader. State: Replica: 7c172b93c2bc4db8bb48da90bcf3eecf, State: Running, Role: LEADER
I20260927 07:11:34.161347 19606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:34.161406 19610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } }
I20260927 07:11:34.161695 19610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c172b93c2bc4db8bb48da90bcf3eecf. Latest consensus state: current_term: 3 leader_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } } }
I20260927 07:11:34.161734 19612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c172b93c2bc4db8bb48da90bcf3eecf" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40505 } } }
I20260927 07:11:34.161866 19610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:34.161870 19612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c172b93c2bc4db8bb48da90bcf3eecf [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:34.162081 19619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:11:34.162935 19619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:11:34.163080 19619 catalog_manager.cc:1259] Loaded cluster ID: 3d1ee5314a0c43949f829d433e23e04a
I20260927 07:11:34.163271 19619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260927 07:11:34.163484 19619 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa21191a9ad google::LogMessage::Fail() at ??:0
@ 0x7fa21191d88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa21191a4de google::LogMessage::Flush() at ??:0
@ 0x7fa21191ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa21983b0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa2198448b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa2137e80d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa2137dd954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa2146416db start_thread at ??:0
@ 0x7fa21041861f clone at ??:0
W20260927 07:11:34.257448 19608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:34.257671 19608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:34.257704 19608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:34.259123 19608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:34.259167 19608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:34.259204 19608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:11:34.260654 19608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:45371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=42347
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:34.260900 19608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:34.261121 19608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:34.266062 19643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:34.266119 19642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:34.266260 19608 server_base.cc:1034] running on GCE node
W20260927 07:11:34.266122 19645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:34.266594 19608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:34.266870 19608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:34.268038 19608 hybrid_clock.cc:648] HybridClock initialized: now 1790493094268020 us; error 29 us; skew 500 ppm
Sep 27 07:11:34 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493094, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:34.270399 19608 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:11:34.270864 19608 webserver.cc:460] Webserver started at http://127.4.174.65:42347/ using document root <none> and password file <none>
I20260927 07:11:34.271070 19608 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:34.271113 19608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:34.272361 19608 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:34.273059 19652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.273238 19608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:34.273356 19608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "eceb0933a873433794d22e355afa18b0"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "3aa68033ebdf92fc77dcbde9325b9b46"
server_key_iv: "ad54dc48a4ae81c3423ed9f4cb4601f5"
server_key_version: "encryptionkey@0"
I20260927 07:11:34.273682 19608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:34.285200 19608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:34.287227 19608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:34.287358 19608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:34.287555 19608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:34.287925 19659 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 07:11:34.289669 19608 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 07:11:34.289709 19608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 07:11:34.289741 19608 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 07:11:34.290345 19608 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 07:11:34.290378 19608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 07:11:34.290468 19659 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Bootstrap starting.
I20260927 07:11:34.292336 19659 log.cc:824] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:34.295214 19659 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:11:34.295507 19659 tablet_bootstrap.cc:492] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Bootstrap complete.
I20260927 07:11:34.295642 19659 ts_tablet_manager.cc:1397] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 07:11:34.296669 19608 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:45371
I20260927 07:11:34.296751 19767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:45371 every 8 connection(s)
I20260927 07:11:34.297039 19608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:11:34.297452 19659 raft_consensus.cc:348] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:34.297804 19659 raft_consensus.cc:729] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eceb0933a873433794d22e355afa18b0, State: Initialized, Role: FOLLOWER
I20260927 07:11:34.298002 19659 consensus_queue.cc:260] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:34.298281 19659 ts_tablet_manager.cc:1428] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:11:34.303755 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 19608
I20260927 07:11:34.304020 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:39353
--local_ip_for_outbound_sockets=127.4.174.66
--tserver_master_addrs=127.4.174.126:40505
--webserver_port=39659
--webserver_interface=127.4.174.66
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:34.387717 19772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:34.387923 19772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:34.387953 19772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:34.388540 19770 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:11:34.297712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:51323)
0927 07:11:34.297897 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.297941 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:34.298074 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:34.388356 (+ 90282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
W20260927 07:11:34.388753 19768 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40505 (0 consecutive failures): Network error: Failed to ping master at 127.4.174.126:40505: Client connection negotiation failed: client connection to 127.4.174.126:40505: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 07:11:34.389065 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.388956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:40659)
0927 07:11:34.389004 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.389046 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 07:11:34.389235 19772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:34.389284 19772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:34.389328 19772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:11:34.389413 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.389329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:35693)
0927 07:11:34.389364 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.389391 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260927 07:11:34.389523 19768 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:11:34.390842 19772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:39353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=39659
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:34.391026 19772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:34.391198 19772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:34.395893 19780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:34.396082 19778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:34.396090 19772 server_base.cc:1034] running on GCE node
W20260927 07:11:34.396219 19777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:34.396399 19772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:34.396588 19772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:34.397715 19772 hybrid_clock.cc:648] HybridClock initialized: now 1790493094397701 us; error 27 us; skew 500 ppm
Sep 27 07:11:34 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493094, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:34.400002 19772 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:11:34.400369 19772 webserver.cc:460] Webserver started at http://127.4.174.66:39659/ using document root <none> and password file <none>
I20260927 07:11:34.400552 19772 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:34.400592 19772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:34.401786 19772 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:34.402312 19787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.402482 19772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.402664 19772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "fc562b57ab3f48008b96bbcb836e4486"
format_stamp: "Formatted at 2026-09-27 07:11:32 on dist-test-slave-zw7p"
server_key: "9c865944a0c3bc87e2dd2d1c515a62fa"
server_key_iv: "26cffcc43b4848d54698b80a0fa277f4"
server_key_version: "encryptionkey@0"
I20260927 07:11:34.403000 19772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:34.416635 19772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:34.418399 19772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:34.418520 19772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:34.418766 19772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:34.419054 19772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:34.419086 19772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.419108 19772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:34.419122 19772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.424355 19772 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:39353
I20260927 07:11:34.424414 19900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:39353 every 8 connection(s)
I20260927 07:11:34.424727 19772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:11:34.425694 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.425339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:36811)
0927 07:11:34.425480 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.425548 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
W20260927 07:11:34.426081 19901 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40505 (0 consecutive failures): Network error: Failed to ping master at 127.4.174.126:40505: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
I20260927 07:11:34.426304 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.426229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:35025)
0927 07:11:34.426264 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.426288 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260927 07:11:34.426605 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.426529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:48847)
0927 07:11:34.426569 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.426590 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260927 07:11:34.426690 19901 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:11:34.429432 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 19772
I20260927 07:11:34.429638 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:32935
--local_ip_for_outbound_sockets=127.4.174.67
--tserver_master_addrs=127.4.174.126:40505
--webserver_port=43009
--webserver_interface=127.4.174.67
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:34.508806 19904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:34.509011 19904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:34.509032 19904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:34.510228 19904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:34.510268 19904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:34.510294 19904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:11:34.511725 19904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:32935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=43009
--tserver_master_addrs=127.4.174.126:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:34.512034 19904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:34.512250 19904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:34.517210 19909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:34.517270 19912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:34.517254 19910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:34.517333 19904 server_base.cc:1034] running on GCE node
I20260927 07:11:34.517599 19904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:34.517833 19904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:34.518976 19904 hybrid_clock.cc:648] HybridClock initialized: now 1790493094518958 us; error 26 us; skew 500 ppm
Sep 27 07:11:34 dist-test-slave-zw7p krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493094, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:34.521263 19904 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:11:34.521644 19904 webserver.cc:460] Webserver started at http://127.4.174.67:43009/ using document root <none> and password file <none>
I20260927 07:11:34.521819 19904 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:34.521863 19904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:34.523121 19904 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:34.523638 19919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.523802 19904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:34.523881 19904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "22cb69a17e744254b6a8988b90adddf0"
format_stamp: "Formatted at 2026-09-27 07:11:33 on dist-test-slave-zw7p"
server_key: "99ac2af4a1e81d1447234f0f70390d9f"
server_key_iv: "64b3ada0fa8dce5cfe04860726adad68"
server_key_version: "encryptionkey@0"
I20260927 07:11:34.524214 19904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:34.534314 19904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:34.536217 19904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:34.536343 19904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:34.536566 19904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:34.536855 19904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:34.536888 19904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.536909 19904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:34.536931 19904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:34.542456 19904 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:32935
I20260927 07:11:34.542510 20032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:32935 every 8 connection(s)
I20260927 07:11:34.542848 19904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:11:34.543818 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.543467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:36275)
0927 07:11:34.543604 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.543676 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260927 07:11:34.544088 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 19904
W20260927 07:11:34.544193 20033 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40505 (0 consecutive failures): Network error: Failed to ping master at 127.4.174.126:40505: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
I20260927 07:11:34.544461 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.544363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:59103)
0927 07:11:34.544408 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.544437 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 07:11:34.544727 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:34.544655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:53995)
0927 07:11:34.544690 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:34.544712 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260927 07:11:34.544842 20033 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:11:35.390116 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:35.389863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:55409)
0927 07:11:35.390023 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:35.390065 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260927 07:11:35.427268 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:35.427048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:58613)
0927 07:11:35.427165 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:35.427206 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260927 07:11:35.545415 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:35.545180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:36797)
0927 07:11:35.545323 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:35.545359 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260927 07:11:35.999553 20040 raft_consensus.cc:482] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:35.999675 20040 raft_consensus.cc:504] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:35.999961 20040 leader_election.cc:288] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935), fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
Sep 27 07:11:35 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493094, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.67@KRBTEST.COM
Sep 27 07:11:36 dist-test-slave-zw7p krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493094, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.66@KRBTEST.COM
I20260927 07:11:36.010924 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:36.000129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:32935 (local address 127.4.174.65:41453)
0927 07:11:36.000230 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:36.000241 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:36.000300 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:36.001270 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:36.001271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:36.001659 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:36.002064 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:36.002067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:36.003214 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:36.003215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:36.004007 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:36.004009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:36.004689 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:36.004690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:36.004820 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:36.005400 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:36.005405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:36.006975 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:36.009237 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:36.009245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:36.009246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:36.009488 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:36.009764 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:36.009765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:36.009765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:36.009897 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:36.010343 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:36.010348 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:36.010478 (+ 130us) client_negotiation.cc:769] Sending connection context
0927 07:11:36.010629 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 07:11:36.010801 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":39,"thread_start_us":47,"threads_started":1}
I20260927 07:11:36.010948 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:36.000267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:41453 (local address 127.4.174.67:32935)
0927 07:11:36.000430 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:36.000441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:36.000480 (+ 39us) server_negotiation.cc:407] Connection header received
0927 07:11:36.000967 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:36.000969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:36.001016 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:36.001144 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:36.002204 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:36.003133 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:36.004093 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:36.004598 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:36.004705 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:36.007106 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:36.007111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:36.007111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:36.007122 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:36.009153 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:36.009570 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:36.009574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:36.009574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:36.009691 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:36.009956 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:36.009956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:36.009957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:36.010282 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:36.010362 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:36.010694 (+ 332us) server_negotiation.cc:299] Negotiation successful
0927 07:11:36.010838 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":61,"threads_started":1}
I20260927 07:11:36.010948 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:36.000424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:42071 (local address 127.4.174.66:39353)
0927 07:11:36.000570 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:36.000578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:36.000595 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:11:36.000998 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:36.001000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:36.001043 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:36.001180 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:36.002020 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:36.003039 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:36.004091 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:36.004603 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:36.004722 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:36.007755 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:36.007760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:36.007760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:36.007774 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:36.009569 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:36.009948 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:36.009951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:36.009951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:36.010054 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:36.010211 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:36.010211 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:36.010212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:36.010482 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:36.010549 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:36.010709 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 07:11:36.010833 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":52,"threads_started":1}
I20260927 07:11:36.010998 20043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:36.000283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39353 (local address 127.4.174.65:42071)
0927 07:11:36.000385 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:36.000394 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:36.000432 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:36.001264 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:36.001267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:36.001659 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:36.001881 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:36.001887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:36.003133 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:36.003134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:36.004007 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:36.004010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:36.004696 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:36.004697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:36.004821 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:36.005359 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:36.005368 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:36.007635 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:36.009655 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:36.009656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:36.009656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:36.009849 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:36.010114 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:36.010115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:36.010115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:36.010163 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:36.010540 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:36.010541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:36.010600 (+ 59us) client_negotiation.cc:769] Sending connection context
0927 07:11:36.010659 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:11:36.010883 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":133,"thread_start_us":51,"threads_started":1}
W20260927 07:11:36.012257 19653 leader_election.cc:341] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353): Not found: Tablet not found: 102fabfd64cc4481951f6d706c06d930
W20260927 07:11:36.012346 19653 leader_election.cc:341] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935): Not found: Tablet not found: 102fabfd64cc4481951f6d706c06d930
I20260927 07:11:36.012382 19653 leader_election.cc:302] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eceb0933a873433794d22e355afa18b0; no voters: 22cb69a17e744254b6a8988b90adddf0, fc562b57ab3f48008b96bbcb836e4486
I20260927 07:11:36.012499 20040 raft_consensus.cc:2726] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:11:36.390795 20043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:36.390657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:60787)
0927 07:11:36.390711 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:36.390751 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 07:11:36.427886 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:36.427678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:45849)
0927 07:11:36.427786 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:36.427828 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260927 07:11:36.546137 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:36.545928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:40355)
0927 07:11:36.546052 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:36.546092 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260927 07:11:37.391420 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:37.391205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:36777)
0927 07:11:37.391335 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:37.391374 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260927 07:11:37.428622 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:37.428379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:33127)
0927 07:11:37.428531 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:37.428576 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260927 07:11:37.546922 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:37.546716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:60363)
0927 07:11:37.546835 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:37.546878 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260927 07:11:37.876278 20050 raft_consensus.cc:482] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:37.876403 20050 raft_consensus.cc:504] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:37.876562 20050 leader_election.cc:288] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935), fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
W20260927 07:11:37.876973 19653 leader_election.cc:341] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353): Not found: Tablet not found: 102fabfd64cc4481951f6d706c06d930
W20260927 07:11:37.877053 19653 leader_election.cc:341] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935): Not found: Tablet not found: 102fabfd64cc4481951f6d706c06d930
I20260927 07:11:37.877075 19653 leader_election.cc:302] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eceb0933a873433794d22e355afa18b0; no voters: 22cb69a17e744254b6a8988b90adddf0, fc562b57ab3f48008b96bbcb836e4486
I20260927 07:11:37.877156 20050 raft_consensus.cc:2726] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:11:38.392066 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:38.391867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:59991)
0927 07:11:38.391976 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:38.392018 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260927 07:11:38.429289 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:38.429097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:54333)
0927 07:11:38.429210 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:38.429245 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260927 07:11:38.547742 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:38.547484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:42629)
0927 07:11:38.547631 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:38.547665 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260927 07:11:39.392695 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:39.392458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:56397)
0927 07:11:39.392594 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:39.392638 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260927 07:11:39.430006 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:39.429781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:51619)
0927 07:11:39.429916 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:39.429963 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260927 07:11:39.548445 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:39.548196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:48925)
0927 07:11:39.548347 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:39.548398 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260927 07:11:40.393402 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:40.393188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:48271)
0927 07:11:40.393306 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:40.393352 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260927 07:11:40.430712 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:40.430438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:45339)
0927 07:11:40.430623 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:40.430665 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260927 07:11:40.549120 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:40.548900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:36417)
0927 07:11:40.549035 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:40.549076 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260927 07:11:40.898845 20060 raft_consensus.cc:482] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:40.898936 20060 raft_consensus.cc:504] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:40.899094 20060 leader_election.cc:288] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935), fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
W20260927 07:11:40.899510 19653 leader_election.cc:341] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935): Not found: Tablet not found: 102fabfd64cc4481951f6d706c06d930
W20260927 07:11:40.899583 19653 leader_election.cc:341] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353): Not found: Tablet not found: 102fabfd64cc4481951f6d706c06d930
I20260927 07:11:40.899613 19653 leader_election.cc:302] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eceb0933a873433794d22e355afa18b0; no voters: 22cb69a17e744254b6a8988b90adddf0, fc562b57ab3f48008b96bbcb836e4486
I20260927 07:11:40.899696 20060 raft_consensus.cc:2726] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:11:41.394109 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:41.393887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:38323)
0927 07:11:41.394011 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:41.394060 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260927 07:11:41.431416 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:41.431192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:32845)
0927 07:11:41.431322 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:41.431364 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260927 07:11:41.549741 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:41.549541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:50313)
0927 07:11:41.549654 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:41.549693 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260927 07:11:42.394928 20064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:42.394682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:33003)
0927 07:11:42.394830 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:42.394879 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260927 07:11:42.432127 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:42.431907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:56709)
0927 07:11:42.432039 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:42.432081 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
I20260927 07:11:42.550526 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:42.550271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:48333)
0927 07:11:42.550435 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:42.550476 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":78,"threads_started":1}
I20260927 07:11:43.395591 20067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:43.395370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:47537)
0927 07:11:43.395499 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:43.395542 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260927 07:11:43.432866 20068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:43.432643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:42025)
0927 07:11:43.432782 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:43.432819 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260927 07:11:43.475235 20069 raft_consensus.cc:482] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:43.475334 20069 raft_consensus.cc:504] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22cb69a17e744254b6a8988b90adddf0" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 32935 } } peers { permanent_uuid: "fc562b57ab3f48008b96bbcb836e4486" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39353 } } peers { permanent_uuid: "eceb0933a873433794d22e355afa18b0" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45371 } }
I20260927 07:11:43.475485 20069 leader_election.cc:288] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935), fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353)
W20260927 07:11:43.475911 19653 leader_election.cc:341] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22cb69a17e744254b6a8988b90adddf0 (127.4.174.67:32935): Not found: Tablet not found: 102fabfd64cc4481951f6d706c06d930
W20260927 07:11:43.475992 19653 leader_election.cc:341] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc562b57ab3f48008b96bbcb836e4486 (127.4.174.66:39353): Not found: Tablet not found: 102fabfd64cc4481951f6d706c06d930
I20260927 07:11:43.476012 19653 leader_election.cc:302] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eceb0933a873433794d22e355afa18b0; no voters: 22cb69a17e744254b6a8988b90adddf0, fc562b57ab3f48008b96bbcb836e4486
I20260927 07:11:43.476081 20069 raft_consensus.cc:2726] T 102fabfd64cc4481951f6d706c06d930 P eceb0933a873433794d22e355afa18b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:11:43.551303 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:43.551022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:44455)
0927 07:11:43.551213 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:43.551245 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":90,"threads_started":1}
I20260927 07:11:44.396256 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:44.396040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:36051)
0927 07:11:44.396165 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:44.396198 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260927 07:11:44.433627 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:44.433389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:44403)
0927 07:11:44.433532 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:44.433574 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
I20260927 07:11:44.552079 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:44.551835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:49935)
0927 07:11:44.551984 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:44.552031 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260927 07:11:45.396993 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:45.396749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:38297)
0927 07:11:45.396897 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:45.396945 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260927 07:11:45.434355 20075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:45.434134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:35661)
0927 07:11:45.434268 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:45.434309 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260927 07:11:45.552847 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:45.552571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:46121)
0927 07:11:45.552746 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:45.552793 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260927 07:11:46.397667 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:46.397460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:33537)
0927 07:11:46.397580 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:46.397619 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260927 07:11:46.435124 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:46.434879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:56401)
0927 07:11:46.435025 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:46.435074 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260927 07:11:46.553575 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:46.553315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:39747)
0927 07:11:46.553477 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:46.553525 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260927 07:11:47.398345 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:47.398107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:45819)
0927 07:11:47.398252 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:47.398296 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":59,"threads_started":1}
I20260927 07:11:47.435895 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:47.435644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:58675)
0927 07:11:47.435795 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:47.435845 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260927 07:11:47.554193 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:47.553996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:36389)
0927 07:11:47.554115 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:47.554147 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260927 07:11:48.399086 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:48.398841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:49041)
0927 07:11:48.398986 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:48.399036 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260927 07:11:48.436651 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:48.436379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:38597)
0927 07:11:48.436530 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:48.436579 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260927 07:11:48.554885 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:48.554674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:37557)
0927 07:11:48.554798 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:48.554837 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260927 07:11:49.399842 20086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:49.399621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.65:47045)
0927 07:11:49.399752 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:49.399795 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260927 07:11:49.437491 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:49.437251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.66:36057)
0927 07:11:49.437411 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:49.437447 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":67,"threads_started":1}
I20260927 07:11:49.552682 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 19608
I20260927 07:11:49.555640 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:49.555442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40505 (local address 127.4.174.67:48689)
0927 07:11:49.555560 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:49.555595 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260927 07:11:49.557063 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 19772
I20260927 07:11:49.560868 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 19904
2026-09-27T07:11:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18077 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:11:49 dist-test-slave-zw7p krb5kdc[20091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:11:49 dist-test-slave-zw7p krb5kdc[20091](info): set up 2 sockets
Sep 27 07:11:49 dist-test-slave-zw7p krb5kdc[20091](info): commencing operation
krb5kdc: starting...
W20260927 07:11:50.518365 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:11:50 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:11:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:11:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:11:50.632700 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40709
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:50.711200 20104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:50.711426 20104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:50.711457 20104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:50.712625 20104 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 07:11:50.712671 20104 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 07:11:50.712687 20104 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 07:11:50.712702 20104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:11:50.712716 20104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:50.712733 20104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:50.712757 20104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:11:50.712781 20104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:11:50.714290 20104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:50.714582 20104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:50.714797 20104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:50.719866 20112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:50.719942 20104 server_base.cc:1034] running on GCE node
W20260927 07:11:50.720005 20110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:50.720109 20109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:50.720499 20104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:50.720818 20104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:50.721963 20104 hybrid_clock.cc:648] HybridClock initialized: now 1790493110721946 us; error 29 us; skew 500 ppm
Sep 27 07:11:50 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493110, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:50.724332 20104 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:11:50.724803 20104 webserver.cc:460] Webserver started at http://127.4.174.126:41175/ using document root <none> and password file <none>
I20260927 07:11:50.724996 20104 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:50.725044 20104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:50.725165 20104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:50.726178 20104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "574ddf87171547f4a31780fd258bfc96"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "1ba5b39104163592bb10c9a2a3aa49db"
server_key_iv: "53754509475ff66c6dd43361a3ce7550"
server_key_version: "encryptionkey@0"
I20260927 07:11:50.726492 20104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "574ddf87171547f4a31780fd258bfc96"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "1ba5b39104163592bb10c9a2a3aa49db"
server_key_iv: "53754509475ff66c6dd43361a3ce7550"
server_key_version: "encryptionkey@0"
I20260927 07:11:50.727705 20104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:11:50.728395 20119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:50.728561 20104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:11:50.728647 20104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "574ddf87171547f4a31780fd258bfc96"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "1ba5b39104163592bb10c9a2a3aa49db"
server_key_iv: "53754509475ff66c6dd43361a3ce7550"
server_key_version: "encryptionkey@0"
I20260927 07:11:50.728706 20104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:50.740710 20104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:50.742931 20104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:50.743069 20104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:50.746773 20104 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40709
I20260927 07:11:50.746835 20171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40709 every 8 connection(s)
I20260927 07:11:50.747210 20104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:11:50.747997 20172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:50.750170 20172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Bootstrap starting.
I20260927 07:11:50.750738 20172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:50.751005 20172 log.cc:824] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:50.751606 20172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: No bootstrap required, opened a new log
I20260927 07:11:50.753134 20172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:11:50.753268 20172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:50.753298 20172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 574ddf87171547f4a31780fd258bfc96, State: Initialized, Role: FOLLOWER
I20260927 07:11:50.753434 20172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:11:50.753494 20172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:11:50.753528 20172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:11:50.753573 20172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:50.754055 20172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:11:50.754141 20172 leader_election.cc:302] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 574ddf87171547f4a31780fd258bfc96; no voters:
I20260927 07:11:50.754298 20172 leader_election.cc:288] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:11:50.754333 20175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:11:50.754462 20175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 1 LEADER]: Becoming Leader. State: Replica: 574ddf87171547f4a31780fd258bfc96, State: Running, Role: LEADER
I20260927 07:11:50.754524 20172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:11:50.754547 20175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:11:50.755124 20177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 574ddf87171547f4a31780fd258bfc96. Latest consensus state: current_term: 1 leader_uuid: "574ddf87171547f4a31780fd258bfc96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } } }
I20260927 07:11:50.755203 20177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:50.755162 20176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "574ddf87171547f4a31780fd258bfc96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } } }
I20260927 07:11:50.755244 20176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: This master's current role is: LEADER
I20260927 07:11:50.755689 20184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:11:50.756155 20184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:11:50.757109 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 20104
I20260927 07:11:50.757215 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:11:50.757392 4793 external_mini_cluster.cc:1428] Setting key 318f99bb2e3c1fb8913ae388898063f1
I20260927 07:11:50.757594 20184 catalog_manager.cc:1347] Generated new cluster ID: de2331e82dd241ada3858f293ca484c3
I20260927 07:11:50.757642 20184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:11:50 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:50.766198 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:50.757986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33188 (local address 127.4.174.126:40709)
0927 07:11:50.758117 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:50.758125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:50.758151 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:11:50.758241 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:50.758246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:50.758286 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:50.758392 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:50.759089 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:50.759985 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:50.760706 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:50.761140 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:50.761233 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:50.763340 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:50.763344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:50.763344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:50.763356 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:50.764808 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:50.765123 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:50.765125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:50.765126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:50.765218 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:50.765389 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:50.765390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:50.765391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:50.765624 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:50.765751 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:50.765862 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 07:11:50.765999 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":69,"threads_started":1}
I20260927 07:11:50.782949 20184 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:11:50.783694 20184 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:11:50.787832 20184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Generated new TSK 0
I20260927 07:11:50.788003 20184 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:11:50.817720 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40709
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:50.930255 20198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:50.930456 20198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:50.930478 20198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:50.931695 20198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:50.931738 20198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:50.931764 20198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:11:50.933225 20198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:50.933444 20198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:50.933666 20198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:50.938436 20206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:50.938581 20198 server_base.cc:1034] running on GCE node
W20260927 07:11:50.938436 20204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:50.938412 20203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:50.938854 20198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:50.939069 20198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:50.940220 20198 hybrid_clock.cc:648] HybridClock initialized: now 1790493110940202 us; error 30 us; skew 500 ppm
Sep 27 07:11:50 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493110, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:50.942677 20198 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:11:50.943099 20198 webserver.cc:460] Webserver started at http://127.4.174.65:38409/ using document root <none> and password file <none>
I20260927 07:11:50.943310 20198 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:50.943358 20198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:50.943477 20198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:50.944536 20198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "2d5eab3dc2b74f6dbc160b95f86ab541"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "688e36ecbfd948094fda3a1e4ac6c00f"
server_key_iv: "f4361d508710c22b03d6e459a1b1df3a"
server_key_version: "encryptionkey@0"
I20260927 07:11:50.944870 20198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "2d5eab3dc2b74f6dbc160b95f86ab541"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "688e36ecbfd948094fda3a1e4ac6c00f"
server_key_iv: "f4361d508710c22b03d6e459a1b1df3a"
server_key_version: "encryptionkey@0"
I20260927 07:11:50.946154 20198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:11:50.946892 20213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:50.947038 20198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:50.947120 20198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "2d5eab3dc2b74f6dbc160b95f86ab541"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "688e36ecbfd948094fda3a1e4ac6c00f"
server_key_iv: "f4361d508710c22b03d6e459a1b1df3a"
server_key_version: "encryptionkey@0"
I20260927 07:11:50.947178 20198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:50.956637 20198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:50.958415 20198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:50.958611 20198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:50.958828 20198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:50.959120 20198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:50.959151 20198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:50.959180 20198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:50.959203 20198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:50.965394 20198 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:37377
I20260927 07:11:50.965428 20326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:37377 every 8 connection(s)
I20260927 07:11:50.965760 20198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:11:50 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493110, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:50.973114 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 20198
I20260927 07:11:50.973222 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:11:50.973474 4793 external_mini_cluster.cc:1428] Setting key 42a41cc695f3622365f0103460ecea25
I20260927 07:11:50.975003 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:50.966428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:58091 (local address 127.4.174.126:40709)
0927 07:11:50.966492 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:50.966494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:50.966687 (+ 193us) server_negotiation.cc:407] Connection header received
0927 07:11:50.967150 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:50.967151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:50.967160 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:50.967232 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:50.968170 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:50.968704 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:50.969672 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:50.969883 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:50.969968 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:50.972013 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:50.972014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:50.972014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:50.972022 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:50.973551 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:50.973959 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:50.973960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:50.973960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:50.974019 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:50.974322 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:50.974329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:50.974329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:50.974481 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:50.974581 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:50.974819 (+ 238us) server_negotiation.cc:299] Negotiation successful
0927 07:11:50.974904 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 07:11:50.975076 20329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:50.966383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:58091)
0927 07:11:50.966537 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:50.966605 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:50.967060 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:50.967295 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:50.967298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:50.967850 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:50.968043 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:50.968048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:50.968784 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:50.968784 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:50.969580 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:50.969583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:50.969979 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:50.969980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:50.970082 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:50.970573 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:50.970584 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:50.971926 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:50.973626 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:50.973634 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:50.973636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:50.973901 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:50.974090 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:50.974090 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:50.974091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:50.974225 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:50.974548 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:50.974572 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:50.974700 (+ 128us) client_negotiation.cc:769] Sending connection context
0927 07:11:50.974780 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 07:11:50.974901 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260927 07:11:50.975525 20327 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40709
I20260927 07:11:50.975615 20327 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:50.975783 20327 heartbeater.cc:511] Master 127.4.174.126:40709 requested a full tablet report, sending...
I20260927 07:11:50.976272 20136 ts_manager.cc:194] Registered new tserver with Master: 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377)
I20260927 07:11:50.976855 20136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:58091
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:11:51.001217 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40709
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:51.081532 20332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:51.081761 20332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:51.081796 20332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:51.083001 20332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:51.083043 20332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:51.083088 20332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:11:51.084547 20332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:51.084779 20332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:51.084992 20332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:51.089929 20338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:51.089923 20337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:51.090077 20340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:51.090170 20332 server_base.cc:1034] running on GCE node
I20260927 07:11:51.090363 20332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:51.090636 20332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:51.091785 20332 hybrid_clock.cc:648] HybridClock initialized: now 1790493111091773 us; error 33 us; skew 500 ppm
Sep 27 07:11:51 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493111, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:51.094352 20332 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:11:51.094851 20332 webserver.cc:460] Webserver started at http://127.4.174.66:35461/ using document root <none> and password file <none>
I20260927 07:11:51.095080 20332 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:51.095127 20332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:51.095219 20332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:51.096254 20332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "6b1af45a20d4458181512553814105da"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "b4dab5ee2313f6dc768298585f3460d9"
server_key_iv: "2cba961a7cec05090df21a5256e9f6ce"
server_key_version: "encryptionkey@0"
I20260927 07:11:51.096581 20332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "6b1af45a20d4458181512553814105da"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "b4dab5ee2313f6dc768298585f3460d9"
server_key_iv: "2cba961a7cec05090df21a5256e9f6ce"
server_key_version: "encryptionkey@0"
I20260927 07:11:51.097905 20332 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 07:11:51.098765 20347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:51.098932 20332 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:11:51.099013 20332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "6b1af45a20d4458181512553814105da"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "b4dab5ee2313f6dc768298585f3460d9"
server_key_iv: "2cba961a7cec05090df21a5256e9f6ce"
server_key_version: "encryptionkey@0"
I20260927 07:11:51.099087 20332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:51.127570 20332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:51.129606 20332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:51.129750 20332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:51.129971 20332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:51.130304 20332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:51.130340 20332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:51.130368 20332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:51.130385 20332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:51.136082 20332 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:36923
I20260927 07:11:51.136153 20460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:36923 every 8 connection(s)
I20260927 07:11:51.136448 20332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
Sep 27 07:11:51 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493111, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:51.145315 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.137140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:35571 (local address 127.4.174.126:40709)
0927 07:11:51.137197 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.137199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.137394 (+ 195us) server_negotiation.cc:407] Connection header received
0927 07:11:51.137897 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.137899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.137911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.137977 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:51.138984 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.139512 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.140478 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.140711 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.140779 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.142883 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:51.142885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:51.142885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:51.142892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:51.144215 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:51.144592 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:51.144593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:51.144593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:51.144639 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:51.144828 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:51.144828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:51.144829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:51.144924 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:51.144967 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.145185 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.145248 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:11:51.145514 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.137129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:35571)
0927 07:11:51.137283 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.137320 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.137780 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.138035 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.138038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.138679 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:51.138884 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.138889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.139584 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.139585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.140386 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.140388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.140787 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.140787 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.140884 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.141372 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:51.141381 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:51.142772 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:51.144302 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:51.144308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:51.144310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:51.144530 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:51.144686 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:51.144687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:51.144687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:51.144785 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:51.144966 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:51.144969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:51.145092 (+ 123us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.145170 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.145292 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260927 07:11:51.145959 20461 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40709
I20260927 07:11:51.146047 20461 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:51.146198 20461 heartbeater.cc:511] Master 127.4.174.126:40709 requested a full tablet report, sending...
I20260927 07:11:51.146251 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 20332
I20260927 07:11:51.146335 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:11:51.146605 4793 external_mini_cluster.cc:1428] Setting key 9ef09fc40939dcf65ca8b272751e4af3
I20260927 07:11:51.146607 20136 ts_manager.cc:194] Registered new tserver with Master: 6b1af45a20d4458181512553814105da (127.4.174.66:36923)
I20260927 07:11:51.147151 20136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:35571
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:11:51.174372 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40709
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:11:51.254029 20466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:11:51.254284 20466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:11:51.254313 20466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:11:51.255612 20466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:11:51.255656 20466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:11:51.255682 20466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:11:51.257110 20466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:11:51.257323 20466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:11:51.257514 20466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:11:51.262265 20474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:51.262290 20466 server_base.cc:1034] running on GCE node
W20260927 07:11:51.262387 20472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:11:51.262521 20471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:11:51.262792 20466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:11:51.263031 20466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:11:51.264171 20466 hybrid_clock.cc:648] HybridClock initialized: now 1790493111264160 us; error 24 us; skew 500 ppm
Sep 27 07:11:51 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493111, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:11:51.266685 20466 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:11:51.267067 20466 webserver.cc:460] Webserver started at http://127.4.174.67:38807/ using document root <none> and password file <none>
I20260927 07:11:51.267253 20466 fs_manager.cc:359] Metadata directory not provided
I20260927 07:11:51.267293 20466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:11:51.267462 20466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:11:51.268488 20466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "d1422f8df4ad410d8efd55ff3b6b969c"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "07640f0b742989ad55c17ea0d6a752aa"
server_key_iv: "20b1f85da09c2d604783abeb06067189"
server_key_version: "encryptionkey@0"
I20260927 07:11:51.268827 20466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "d1422f8df4ad410d8efd55ff3b6b969c"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "07640f0b742989ad55c17ea0d6a752aa"
server_key_iv: "20b1f85da09c2d604783abeb06067189"
server_key_version: "encryptionkey@0"
I20260927 07:11:51.270081 20466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:11:51.270841 20481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:51.270995 20466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:11:51.271076 20466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "d1422f8df4ad410d8efd55ff3b6b969c"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "07640f0b742989ad55c17ea0d6a752aa"
server_key_iv: "20b1f85da09c2d604783abeb06067189"
server_key_version: "encryptionkey@0"
I20260927 07:11:51.271149 20466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:11:51.291711 20466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:11:51.293653 20466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:11:51.293787 20466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:11:51.293994 20466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:11:51.294297 20466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:11:51.294330 20466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:51.294360 20466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:11:51.294385 20466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:11:51.299827 20466 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:37195
I20260927 07:11:51.299890 20594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:37195 every 8 connection(s)
I20260927 07:11:51.300172 20466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:11:51 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493111, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:11:51.308736 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.300841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:34973 (local address 127.4.174.126:40709)
0927 07:11:51.300893 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.300895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.301076 (+ 181us) server_negotiation.cc:407] Connection header received
0927 07:11:51.301580 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.301582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.301594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.301656 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:51.302640 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.303193 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.304166 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.304367 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.304420 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.306412 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:51.306414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:51.306414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:51.306423 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:51.307524 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:51.307875 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:51.307876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:51.307876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:51.307935 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:51.308137 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:51.308138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:51.308138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:51.308253 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:51.308306 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.308556 (+ 250us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.308643 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 07:11:51.308897 20597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.300818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:34973)
0927 07:11:51.300959 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.300995 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.301484 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.301749 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.301753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.302323 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:11:51.302515 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.302520 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.303284 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.303284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.304076 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.304078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.304435 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.304436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.304551 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.305011 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:11:51.305020 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:11:51.306325 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:11:51.307592 (+ 1267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:51.307598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:51.307601 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:51.307811 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:51.307986 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:11:51.307987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:11:51.307987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:11:51.308083 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:11:51.308303 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:11:51.308309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:11:51.308441 (+ 132us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.308526 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.308679 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260927 07:11:51.309245 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 20466
I20260927 07:11:51.309335 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:11:51.309445 20595 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40709
I20260927 07:11:51.309530 4793 external_mini_cluster.cc:1428] Setting key 2d4e25215e03a3877feb548afc8d7880
I20260927 07:11:51.309540 20595 heartbeater.cc:464] Registering TS with master...
I20260927 07:11:51.309732 20595 heartbeater.cc:511] Master 127.4.174.126:40709 requested a full tablet report, sending...
I20260927 07:11:51.310089 20136 ts_manager.cc:194] Registered new tserver with Master: d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195)
I20260927 07:11:51.310487 20136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:34973
I20260927 07:11:51.321128 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:11:51.328006 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.322437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.4.174.126:40709)
0927 07:11:51.322504 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.322507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.322659 (+ 152us) server_negotiation.cc:407] Connection header received
0927 07:11:51.322705 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.322706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.322715 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.322763 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:11:51.323271 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.323836 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.324531 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.324716 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.324767 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.325829 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:11:51.325830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:11:51.325831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:11:51.325837 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:11:51.326959 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:51.327317 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:51.327318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:51.327318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:51.327372 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:11:51.327565 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:11:51.327566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:11:51.327566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:11:51.327705 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:11:51.327778 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.327858 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.327913 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:11:51.329535 20136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:11:51.331166 20136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:11:51.337452 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.333716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.4.174.65:37377)
0927 07:11:51.333893 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.333897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.333906 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:11:51.333938 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.333940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.333954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.334222 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:51.334293 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.335359 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.336599 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.337078 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.337140 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.337218 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.337330 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.337370 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
I20260927 07:11:51.337477 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.333521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:37377 (local address 127.0.0.1:45984)
0927 07:11:51.333732 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.333741 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.333795 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.334099 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.334099 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.334102 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:51.334147 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.334148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.335448 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.335449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.336530 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.336532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.337151 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.337152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.337240 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.337255 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.337317 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.337370 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260927 07:11:51.337530 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.333391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.4.174.67:37195)
0927 07:11:51.333559 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.333565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.333732 (+ 167us) server_negotiation.cc:407] Connection header received
0927 07:11:51.333853 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.333855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.333871 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.333920 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:51.334144 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.334967 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.336405 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.337077 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.337143 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.337218 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.337417 (+ 199us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.337460 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260927 07:11:51.337568 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.333311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36923 (local address 127.0.0.1:47030)
0927 07:11:51.333634 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.333674 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.333742 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.333972 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.333977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.333979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:51.334047 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.334049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.335389 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.335390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.336276 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.336278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.337256 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.337258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.337332 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.337334 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.337378 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.337417 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":1,"thread_start_us":56,"threads_started":1}
I20260927 07:11:51.337625 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.333475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.4.174.66:36923)
0927 07:11:51.333622 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.333626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.333829 (+ 203us) server_negotiation.cc:407] Connection header received
0927 07:11:51.333871 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.333873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.333888 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.333959 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:51.334126 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.335281 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.336345 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.337076 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.337143 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.337218 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.337515 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.337555 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260927 07:11:51.337769 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.333312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37195 (local address 127.0.0.1:38254)
0927 07:11:51.333634 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.333674 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.333756 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.334016 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.334016 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.334019 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:51.334067 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.334068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.335056 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.335057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.336301 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.336303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.337255 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.337256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.337327 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.337332 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.337665 (+ 333us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.337707 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":38,"thread_start_us":67,"threads_started":1}
I20260927 07:11:51.338519 20529 tablet_service.cc:1467] Processing CreateTablet for tablet 5485c071853b44da8a9b87d074634139 (DEFAULT_TABLE table=test-table [id=d77f4cd2c3dc448c9d639b307f74e0ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:51.338728 20395 tablet_service.cc:1467] Processing CreateTablet for tablet 5485c071853b44da8a9b87d074634139 (DEFAULT_TABLE table=test-table [id=d77f4cd2c3dc448c9d639b307f74e0ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:51.338918 20529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5485c071853b44da8a9b87d074634139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:51.338878 20261 tablet_service.cc:1467] Processing CreateTablet for tablet 5485c071853b44da8a9b87d074634139 (DEFAULT_TABLE table=test-table [id=d77f4cd2c3dc448c9d639b307f74e0ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:11:51.339075 20395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5485c071853b44da8a9b87d074634139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:51.339133 20261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5485c071853b44da8a9b87d074634139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:11:51.341369 20612 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Bootstrap starting.
I20260927 07:11:51.341960 20612 tablet_bootstrap.cc:654] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:51.342193 20613 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Bootstrap starting.
I20260927 07:11:51.342694 20614 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Bootstrap starting.
I20260927 07:11:51.342689 20612 log.cc:824] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:51.342891 20613 tablet_bootstrap.cc:654] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:51.343266 20614 tablet_bootstrap.cc:654] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Neither blocks nor log segments found. Creating new log.
I20260927 07:11:51.343385 20613 log.cc:824] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:51.343480 20612 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: No bootstrap required, opened a new log
I20260927 07:11:51.343545 20612 ts_tablet_manager.cc:1397] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:11:51.343755 20614 log.cc:824] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Log is configured to *not* fsync() on all Append() calls
I20260927 07:11:51.344376 20614 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: No bootstrap required, opened a new log
I20260927 07:11:51.344434 20614 ts_tablet_manager.cc:1397] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:11:51.345084 20612 raft_consensus.cc:348] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.345234 20612 raft_consensus.cc:374] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:51.345258 20612 raft_consensus.cc:729] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1422f8df4ad410d8efd55ff3b6b969c, State: Initialized, Role: FOLLOWER
I20260927 07:11:51.345376 20612 consensus_queue.cc:260] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.345639 20612 ts_tablet_manager.cc:1428] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:11:51.345659 20595 heartbeater.cc:503] Master 127.4.174.126:40709 was elected leader, sending a full tablet report...
I20260927 07:11:51.346223 20613 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: No bootstrap required, opened a new log
I20260927 07:11:51.346283 20613 ts_tablet_manager.cc:1397] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260927 07:11:51.346436 20614 raft_consensus.cc:348] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.346666 20614 raft_consensus.cc:374] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:51.346726 20614 raft_consensus.cc:729] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5eab3dc2b74f6dbc160b95f86ab541, State: Initialized, Role: FOLLOWER
I20260927 07:11:51.346843 20614 consensus_queue.cc:260] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.347148 20614 ts_tablet_manager.cc:1428] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:11:51.347169 20327 heartbeater.cc:503] Master 127.4.174.126:40709 was elected leader, sending a full tablet report...
I20260927 07:11:51.348080 20613 raft_consensus.cc:348] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.348212 20613 raft_consensus.cc:374] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:11:51.348240 20613 raft_consensus.cc:729] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1af45a20d4458181512553814105da, State: Initialized, Role: FOLLOWER
I20260927 07:11:51.348349 20613 consensus_queue.cc:260] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.348575 20613 ts_tablet_manager.cc:1428] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:11:51.348606 20461 heartbeater.cc:503] Master 127.4.174.126:40709 was elected leader, sending a full tablet report...
W20260927 07:11:51.387081 20462 tablet.cc:2369] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:11:51.466621 20328 tablet.cc:2369] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:11:51.549749 20619 raft_consensus.cc:482] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:51.549911 20619 raft_consensus.cc:504] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.550284 20619 leader_election.cc:288] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b1af45a20d4458181512553814105da (127.4.174.66:36923), d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195)
W20260927 07:11:51.550740 20596 tablet.cc:2369] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:11:51.551014 20620 raft_consensus.cc:482] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:11:51.551136 20620 raft_consensus.cc:504] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.551406 20620 leader_election.cc:288] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377), d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195)
I20260927 07:11:51.554165 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.550502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:51861 (local address 127.4.174.66:36923)
0927 07:11:51.550600 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.550602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.550679 (+ 77us) server_negotiation.cc:407] Connection header received
0927 07:11:51.550778 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.550779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.550798 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.550867 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:51.551090 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.551845 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.553225 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.553777 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.553849 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.553858 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.554053 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.554100 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":32}
I20260927 07:11:51.554253 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.550592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:52197 (local address 127.4.174.67:37195)
0927 07:11:51.550641 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.550643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.550709 (+ 66us) server_negotiation.cc:407] Connection header received
0927 07:11:51.550819 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.550821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.550834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.550879 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:51.551131 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.551845 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.553294 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.553851 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.553901 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.553909 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.554128 (+ 219us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.554165 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:11:51.554499 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.550474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37195 (local address 127.4.174.65:52197)
0927 07:11:51.550650 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.550659 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.550709 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.550929 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.550930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.550932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:51.551035 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.551038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.551965 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.551966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.553197 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.553200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.553973 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.553974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.554048 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.554052 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.554353 (+ 301us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.554399 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":14,"thread_start_us":111,"threads_started":1}
I20260927 07:11:51.554713 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.550447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36923 (local address 127.4.174.65:51861)
0927 07:11:51.550605 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.550615 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.550681 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.550925 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.550927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.550930 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:51.551009 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.551016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.551924 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.551925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.553140 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.553142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.553903 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.553904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.553976 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.553981 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.554484 (+ 503us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.554527 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":60,"threads_started":1}
I20260927 07:11:51.554967 20549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5485c071853b44da8a9b87d074634139" candidate_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" is_pre_election: true
I20260927 07:11:51.555111 20549 raft_consensus.cc:2454] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d5eab3dc2b74f6dbc160b95f86ab541 in term 0.
I20260927 07:11:51.555179 20415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5485c071853b44da8a9b87d074634139" candidate_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b1af45a20d4458181512553814105da" is_pre_election: true
I20260927 07:11:51.555287 20415 raft_consensus.cc:2454] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d5eab3dc2b74f6dbc160b95f86ab541 in term 0.
I20260927 07:11:51.555411 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.551908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:42343 (local address 127.4.174.67:37195)
0927 07:11:51.552065 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.552067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.552074 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:11:51.552095 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.552096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.552106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.552175 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:51.552374 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.553112 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.554394 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.554986 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.555065 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.555074 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.555312 (+ 238us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.555349 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260927 07:11:51.555426 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.551516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:37377 (local address 127.4.174.66:56659)
0927 07:11:51.551616 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.551625 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.551660 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.552025 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.552026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.552027 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:51.552084 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.552085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.552920 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.552921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.554061 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.554063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.555196 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.555197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.555269 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.555273 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.555329 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.555365 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260927 07:11:51.555423 20215 leader_election.cc:302] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541, d1422f8df4ad410d8efd55ff3b6b969c; no voters:
I20260927 07:11:51.555641 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.551721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:56659 (local address 127.4.174.65:37377)
0927 07:11:51.551771 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.551773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.551779 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:11:51.551798 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.551799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.551807 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.551848 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:11:51.552156 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.552838 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.554145 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.554679 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.554740 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.554747 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.555515 (+ 768us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.555555 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 07:11:51.555655 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.551531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37195 (local address 127.4.174.66:42343)
0927 07:11:51.551779 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.551788 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.551826 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.552220 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:11:51.552221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:11:51.552223 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:11:51.552294 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.552295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.553208 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.553209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.554304 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:11:51.554307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.555090 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.555091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:11:51.555161 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.555165 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:11:51.555214 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 07:11:51.555250 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":77,"threads_started":1}
I20260927 07:11:51.555822 20281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5485c071853b44da8a9b87d074634139" candidate_uuid: "6b1af45a20d4458181512553814105da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" is_pre_election: true
I20260927 07:11:51.555577 20619 raft_consensus.cc:2781] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:11:51.556069 20619 raft_consensus.cc:482] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:11:51.556035 20549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5485c071853b44da8a9b87d074634139" candidate_uuid: "6b1af45a20d4458181512553814105da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" is_pre_election: true
I20260927 07:11:51.556107 20619 raft_consensus.cc:3037] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:51.556111 20549 raft_consensus.cc:2454] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b1af45a20d4458181512553814105da in term 0.
I20260927 07:11:51.556268 20349 leader_election.cc:302] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b1af45a20d4458181512553814105da, d1422f8df4ad410d8efd55ff3b6b969c; no voters:
I20260927 07:11:51.556394 20620 raft_consensus.cc:2781] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:11:51.556465 20620 raft_consensus.cc:482] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:11:51.556501 20620 raft_consensus.cc:3037] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:51.556934 20619 raft_consensus.cc:504] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.557088 20281 raft_consensus.cc:2379] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b1af45a20d4458181512553814105da in current term 1: Already voted for candidate 2d5eab3dc2b74f6dbc160b95f86ab541 in this term.
I20260927 07:11:51.557219 20415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5485c071853b44da8a9b87d074634139" candidate_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b1af45a20d4458181512553814105da"
I20260927 07:11:51.557267 20549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5485c071853b44da8a9b87d074634139" candidate_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1422f8df4ad410d8efd55ff3b6b969c"
I20260927 07:11:51.557281 20620 raft_consensus.cc:504] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.557327 20549 raft_consensus.cc:3037] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:11:51.557401 20620 leader_election.cc:288] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 election: Requested vote from peers 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377), d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195)
I20260927 07:11:51.557402 20415 raft_consensus.cc:2379] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d5eab3dc2b74f6dbc160b95f86ab541 in current term 1: Already voted for candidate 6b1af45a20d4458181512553814105da in this term.
I20260927 07:11:51.557571 20281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5485c071853b44da8a9b87d074634139" candidate_uuid: "6b1af45a20d4458181512553814105da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541"
I20260927 07:11:51.557638 20281 raft_consensus.cc:2379] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b1af45a20d4458181512553814105da in current term 1: Already voted for candidate 2d5eab3dc2b74f6dbc160b95f86ab541 in this term.
I20260927 07:11:51.558082 20549 raft_consensus.cc:2454] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d5eab3dc2b74f6dbc160b95f86ab541 in term 1.
I20260927 07:11:51.558218 20215 leader_election.cc:302] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541, d1422f8df4ad410d8efd55ff3b6b969c; no voters: 6b1af45a20d4458181512553814105da
I20260927 07:11:51.558226 20619 leader_election.cc:288] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 1 election: Requested vote from peers 6b1af45a20d4458181512553814105da (127.4.174.66:36923), d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195)
I20260927 07:11:51.558240 20549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5485c071853b44da8a9b87d074634139" candidate_uuid: "6b1af45a20d4458181512553814105da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1422f8df4ad410d8efd55ff3b6b969c"
I20260927 07:11:51.558291 20619 raft_consensus.cc:2781] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:11:51.558315 20549 raft_consensus.cc:2379] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b1af45a20d4458181512553814105da in current term 1: Already voted for candidate 2d5eab3dc2b74f6dbc160b95f86ab541 in this term.
I20260927 07:11:51.558322 20619 raft_consensus.cc:686] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 LEADER]: Becoming Leader. State: Replica: 2d5eab3dc2b74f6dbc160b95f86ab541, State: Running, Role: LEADER
I20260927 07:11:51.558396 20619 consensus_queue.cc:237] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:11:51.558507 20349 leader_election.cc:302] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b1af45a20d4458181512553814105da; no voters: 2d5eab3dc2b74f6dbc160b95f86ab541, d1422f8df4ad410d8efd55ff3b6b969c
I20260927 07:11:51.558610 20620 raft_consensus.cc:2726] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260927 07:11:51.559252 20136 catalog_manager.cc:5495] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } health_report { overall_health: UNKNOWN } } }
I20260927 07:11:51.565471 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.563369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45990 (local address 127.4.174.65:37377)
0927 07:11:51.563428 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:11:51.563430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:11:51.563438 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:11:51.563465 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:11:51.563466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:11:51.563482 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:11:51.563541 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:11:51.563640 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.564093 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.564799 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:11:51.564991 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:11:51.565043 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:11:51.565187 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:11:51.565285 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:11:51.565353 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:11:51.565386 (+ 33us) server_negotiation.cc:299] Negotiation successful
0927 07:11:51.565405 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 07:11:51.567075 20415 raft_consensus.cc:1261] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Refusing update from remote peer 2d5eab3dc2b74f6dbc160b95f86ab541: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:51.567106 20549 raft_consensus.cc:1261] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 1 FOLLOWER]: Refusing update from remote peer 2d5eab3dc2b74f6dbc160b95f86ab541: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:11:51.567363 20625 consensus_queue.cc:1035] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:51.567538 20619 consensus_queue.cc:1035] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:11:51.569870 20629 mvcc.cc:205] Tried to move back new op lower bound from 7333859784976576512 to 7333859784944041984. Current Snapshot: MvccSnapshot[applied={T|T < 7333859784976576512}]
I20260927 07:11:51.570212 20630 mvcc.cc:205] Tried to move back new op lower bound from 7333859784976576512 to 7333859784944041984. Current Snapshot: MvccSnapshot[applied={T|T < 7333859784976576512 or (T in {7333859784976576512})}]
I20260927 07:11:51.572388 20136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:11:51.572511 20136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:11:51.573348 20136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Sending DeleteTablet for 3 replicas of tablet 5485c071853b44da8a9b87d074634139
I20260927 07:11:51.573776 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 20198
I20260927 07:11:51.573773 20529 tablet_service.cc:1514] Processing DeleteTablet for tablet 5485c071853b44da8a9b87d074634139 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:51 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:38254
I20260927 07:11:51.573791 20395 tablet_service.cc:1514] Processing DeleteTablet for tablet 5485c071853b44da8a9b87d074634139 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:51 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:47030
W20260927 07:11:51.578785 20122 connection.cc:535] client connection to 127.4.174.65:37377 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 07:11:51.578890 20122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 07:11:51.578887 20637 tablet_replica.cc:330] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: stopping tablet replica
I20260927 07:11:51.578977 20637 raft_consensus.cc:2229] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:11:51.579034 20638 tablet_replica.cc:330] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: stopping tablet replica
I20260927 07:11:51.579082 20637 raft_consensus.cc:2258] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:11:51.579133 20638 raft_consensus.cc:2229] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:11:51.579258 20638 raft_consensus.cc:2258] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 07:11:51.579504 20606 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:11:51.579314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:37377 (local address 127.0.0.1:46006)
0927 07:11:51.579367 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.579378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:11:51.579417 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:11:51.579456 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:37377: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 07:11:51.579599 20637 ts_tablet_manager.cc:1905] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 07:11:51.579705 20122 catalog_manager.cc:4584] TS 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5485c071853b44da8a9b87d074634139: Network error: Client connection negotiation failed: client connection to 127.4.174.65:37377: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260927 07:11:51.579759 20638 ts_tablet_manager.cc:1905] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:11:51.579768 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 20332
I20260927 07:11:51.582674 20637 ts_tablet_manager.cc:1918] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:11:51.582734 20637 log.cc:1196] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/5485c071853b44da8a9b87d074634139
I20260927 07:11:51.583189 20637 ts_tablet_manager.cc:1939] T 5485c071853b44da8a9b87d074634139 P d1422f8df4ad410d8efd55ff3b6b969c: Deleting consensus metadata
I20260927 07:11:51.583561 20121 catalog_manager.cc:4841] TS d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): tablet 5485c071853b44da8a9b87d074634139 (table test-table [id=d77f4cd2c3dc448c9d639b307f74e0ae]) successfully deleted
I20260927 07:11:51.583933 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 20466
I20260927 07:11:51.583988 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:11:51.583862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36923 (local address 127.0.0.1:47046)
0927 07:11:51.583912 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 07:11:51.583959 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.66:36923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 07:11:51.588161 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 20104
I20260927 07:12:11.592774 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40709
--webserver_interface=127.4.174.126
--webserver_port=41175
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:11.670771 20640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:11.670970 20640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:11.670990 20640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:11.672106 20640 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 07:12:11.672147 20640 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 07:12:11.672158 20640 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 07:12:11.672169 20640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:11.672194 20640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:11.672209 20640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:11.672235 20640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:11.672256 20640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:11.673699 20640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:11.673920 20640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:11.674136 20640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:11.682518 20646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:11.682614 20640 server_base.cc:1034] running on GCE node
W20260927 07:12:11.682545 20647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:11.682534 20649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:11.682901 20640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:11.683218 20640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:11.684355 20640 hybrid_clock.cc:648] HybridClock initialized: now 1790493131684346 us; error 30 us; skew 500 ppm
Sep 27 07:12:11 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493131, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:11.692961 20640 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:12:11.693485 20640 webserver.cc:460] Webserver started at http://127.4.174.126:41175/ using document root <none> and password file <none>
I20260927 07:12:11.693734 20640 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:11.693799 20640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:11.695470 20640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260927 07:12:11.696147 20656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:11.696331 20640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:11.696422 20640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "574ddf87171547f4a31780fd258bfc96"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "1ba5b39104163592bb10c9a2a3aa49db"
server_key_iv: "53754509475ff66c6dd43361a3ce7550"
server_key_version: "encryptionkey@0"
I20260927 07:12:11.696846 20640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:11.710788 20640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:11.712620 20640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:11.712731 20640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:11.716475 20640 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40709
I20260927 07:12:11.716521 20708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40709 every 8 connection(s)
I20260927 07:12:11.716902 20640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:11.717710 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 20640
I20260927 07:12:11.718043 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:37377
--local_ip_for_outbound_sockets=127.4.174.65
--tserver_master_addrs=127.4.174.126:40709
--webserver_port=38409
--webserver_interface=127.4.174.65
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:12:11.718813 20709 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:12:11.719694 20709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Bootstrap starting.
I20260927 07:12:11.728094 20709 log.cc:824] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:11.730274 20709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:12:11.730647 20709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Bootstrap complete.
I20260927 07:12:11.732882 20709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:12:11.733124 20709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 574ddf87171547f4a31780fd258bfc96, State: Initialized, Role: FOLLOWER
I20260927 07:12:11.733278 20709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:12:11.733345 20709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:11.733393 20709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:11.733443 20709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:12:11.734515 20709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:12:11.734656 20709 leader_election.cc:302] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 574ddf87171547f4a31780fd258bfc96; no voters:
I20260927 07:12:11.734845 20709 leader_election.cc:288] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 07:12:11.734894 20713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 07:12:11.735026 20713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 2 LEADER]: Becoming Leader. State: Replica: 574ddf87171547f4a31780fd258bfc96, State: Running, Role: LEADER
I20260927 07:12:11.735114 20709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:11.735196 20713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:12:11.735575 20713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 574ddf87171547f4a31780fd258bfc96. Latest consensus state: current_term: 2 leader_uuid: "574ddf87171547f4a31780fd258bfc96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } } }
I20260927 07:12:11.735764 20715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "574ddf87171547f4a31780fd258bfc96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } } }
I20260927 07:12:11.736040 20715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:11.736230 20713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:11.736532 20725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:11.737478 20725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:11.737597 20725 catalog_manager.cc:1259] Loaded cluster ID: de2331e82dd241ada3858f293ca484c3
I20260927 07:12:11.737634 20725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:11.738840 20725 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:11.739025 20725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Loaded TSK: 0
I20260927 07:12:11.746711 20725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Generated new TSK 1
I20260927 07:12:11.746824 20725 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 07:12:11.799012 20711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:11.799261 20711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:11.799299 20711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:11.800489 20711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:11.800535 20711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:11.800570 20711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:12:11.802001 20711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:37377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=38409
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:11.802228 20711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:11.802445 20711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:11.807085 20739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:11.807101 20737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:11.807327 20711 server_base.cc:1034] running on GCE node
W20260927 07:12:11.807101 20736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:11.807582 20711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:11.807799 20711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:11.808936 20711 hybrid_clock.cc:648] HybridClock initialized: now 1790493131808923 us; error 30 us; skew 500 ppm
Sep 27 07:12:11 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493131, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:11.811344 20711 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:12:11.811749 20711 webserver.cc:460] Webserver started at http://127.4.174.65:38409/ using document root <none> and password file <none>
I20260927 07:12:11.811969 20711 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:11.812021 20711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:11.813290 20711 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:12:11.813851 20746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:11.814023 20711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:12:11.814123 20711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "2d5eab3dc2b74f6dbc160b95f86ab541"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "688e36ecbfd948094fda3a1e4ac6c00f"
server_key_iv: "f4361d508710c22b03d6e459a1b1df3a"
server_key_version: "encryptionkey@0"
I20260927 07:12:11.814527 20711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:11.827498 20711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:11.829634 20711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:11.829769 20711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:11.829993 20711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:11.830420 20753 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 07:12:11.831848 20711 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 07:12:11.831904 20711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 07:12:11.831935 20711 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 07:12:11.832738 20711 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 07:12:11.832779 20711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 07:12:11.832854 20753 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Bootstrap starting.
I20260927 07:12:11.839625 20711 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:37377
I20260927 07:12:11.839706 20860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:37377 every 8 connection(s)
I20260927 07:12:11.840090 20711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:12:11.842278 20753 log.cc:824] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:11.843150 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 20711
I20260927 07:12:11.843466 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:36923
--local_ip_for_outbound_sockets=127.4.174.66
--tserver_master_addrs=127.4.174.126:40709
--webserver_port=35461
--webserver_interface=127.4.174.66
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:12:11.844722 20753 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:12:11.844993 20753 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Bootstrap complete.
I20260927 07:12:11.846066 20753 ts_tablet_manager.cc:1397] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.003s
I20260927 07:12:11.847241 20753 raft_consensus.cc:348] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:12:11.847829 20753 raft_consensus.cc:729] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5eab3dc2b74f6dbc160b95f86ab541, State: Initialized, Role: FOLLOWER
I20260927 07:12:11.847980 20753 consensus_queue.cc:260] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:12:11.848253 20753 ts_tablet_manager.cc:1428] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Sep 27 07:12:11 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493131, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:11.853070 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:11.840927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:54101)
0927 07:12:11.841109 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:11.841156 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:11.842046 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:11.842514 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:11.842518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:11.842893 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:11.843168 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:11.843175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.844824 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.844825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:11.846046 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:11.846049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.846902 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.846903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:11.847031 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:11.847548 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:11.847557 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:11.849167 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:11.851286 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:11.851294 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:11.851296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:11.851579 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:11.851863 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:11.851864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:11.851864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:11.851993 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:11.852426 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:11.852432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:11.852587 (+ 155us) client_negotiation.cc:769] Sending connection context
0927 07:12:11.852701 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 07:12:11.852868 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260927 07:12:11.853098 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:11.841326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:54101 (local address 127.4.174.126:40709)
0927 07:12:11.841524 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:11.841533 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:11.841585 (+ 52us) server_negotiation.cc:407] Connection header received
0927 07:12:11.842271 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:11.842278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:11.842337 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:11.842478 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:11.843302 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.844694 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.846144 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.846797 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.846912 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:11.849280 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:11.849284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:11.849284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:11.849296 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:11.851184 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:11.851663 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:11.851666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:11.851666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:11.851789 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:11.852062 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:11.852063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:11.852063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:11.852351 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:11.852438 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:11.852777 (+ 339us) server_negotiation.cc:299] Negotiation successful
0927 07:12:11.852914 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260927 07:12:11.854628 20861 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40709
I20260927 07:12:11.854728 20861 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:11.854905 20861 heartbeater.cc:511] Master 127.4.174.126:40709 requested a full tablet report, sending...
I20260927 07:12:11.855719 20673 ts_manager.cc:194] Registered new tserver with Master: 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377)
I20260927 07:12:11.856941 20673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:54101
I20260927 07:12:11.857969 20861 heartbeater.cc:503] Master 127.4.174.126:40709 was elected leader, sending a full tablet report...
I20260927 07:12:11.862681 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:11.856590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.4.174.65:37377)
0927 07:12:11.858670 (+ 2080us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:11.858675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:11.858723 (+ 48us) server_negotiation.cc:407] Connection header received
0927 07:12:11.858812 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:11.858814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:11.858830 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:11.858913 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:11.859064 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.860063 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.861571 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.862264 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.862336 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:11.862386 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:11.862548 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 07:12:11.862613 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2044,"thread_start_us":64,"threads_started":1}
I20260927 07:12:11.862696 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:11.856346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:37377 (local address 127.0.0.1:35056)
0927 07:12:11.858642 (+ 2296us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:11.858677 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:11.858742 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:11.858918 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:11.858920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:11.858922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:11.859004 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:11.859006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.860156 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.860157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:11.861485 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:11.861488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.862355 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.862356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:11.862452 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:11.862466 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 07:12:11.862530 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 07:12:11.862613 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2267,"thread_start_us":90,"threads_started":1}
I20260927 07:12:11.863217 20795 tablet_service.cc:1514] Processing DeleteTablet for tablet 5485c071853b44da8a9b87d074634139 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:51 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:35056
I20260927 07:12:11.863788 20870 tablet_replica.cc:330] stopping tablet replica
I20260927 07:12:11.863890 20870 raft_consensus.cc:2229] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:11.863951 20870 raft_consensus.cc:2258] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:11.864243 20870 ts_tablet_manager.cc:1905] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:11.866425 20870 ts_tablet_manager.cc:1918] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:11.866500 20870 log.cc:1196] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal/wals/5485c071853b44da8a9b87d074634139
I20260927 07:12:11.866894 20870 ts_tablet_manager.cc:1939] T 5485c071853b44da8a9b87d074634139 P 2d5eab3dc2b74f6dbc160b95f86ab541: Deleting consensus metadata
I20260927 07:12:11.867430 20659 catalog_manager.cc:4841] TS 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377): tablet 5485c071853b44da8a9b87d074634139 (table test-table [id=d77f4cd2c3dc448c9d639b307f74e0ae]) successfully deleted
W20260927 07:12:11.929916 20866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:11.930121 20866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:11.930152 20866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:11.931280 20866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:11.931319 20866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:11.931352 20866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:12:11.932729 20866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:36923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=35461
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:11.932960 20866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:11.933161 20866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:11.938015 20878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:11.938102 20875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:11.938014 20876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:11.938206 20866 server_base.cc:1034] running on GCE node
I20260927 07:12:11.938458 20866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:11.938717 20866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:11.939857 20866 hybrid_clock.cc:648] HybridClock initialized: now 1790493131939841 us; error 38 us; skew 500 ppm
Sep 27 07:12:11 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493131, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:11.942220 20866 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:12:11.942664 20866 webserver.cc:460] Webserver started at http://127.4.174.66:35461/ using document root <none> and password file <none>
I20260927 07:12:11.942853 20866 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:11.942911 20866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:11.944202 20866 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:11.944825 20885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:11.944985 20866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:11.945071 20866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "6b1af45a20d4458181512553814105da"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "b4dab5ee2313f6dc768298585f3460d9"
server_key_iv: "2cba961a7cec05090df21a5256e9f6ce"
server_key_version: "encryptionkey@0"
I20260927 07:12:11.945411 20866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:11.954155 20866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:11.956091 20866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:11.956205 20866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:11.956369 20866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:11.956693 20892 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 07:12:11.958048 20866 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 07:12:11.958086 20866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260927 07:12:11.958112 20866 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 07:12:11.958771 20866 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 07:12:11.958802 20866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:11.958877 20892 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Bootstrap starting.
I20260927 07:12:11.960455 20892 log.cc:824] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:11.963439 20892 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:12:11.963776 20892 tablet_bootstrap.cc:492] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Bootstrap complete.
I20260927 07:12:11.963923 20892 ts_tablet_manager.cc:1397] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 07:12:11.964869 20866 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:36923
I20260927 07:12:11.964937 21000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:36923 every 8 connection(s)
I20260927 07:12:11.965231 20866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:12:11.965730 20892 raft_consensus.cc:348] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:12:11.965982 20892 raft_consensus.cc:729] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1af45a20d4458181512553814105da, State: Initialized, Role: FOLLOWER
I20260927 07:12:11.966105 20892 consensus_queue.cc:260] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } }
I20260927 07:12:11.966339 20892 ts_tablet_manager.cc:1428] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Sep 27 07:12:11 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493131, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:11.972368 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 20866
I20260927 07:12:11.972641 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:37195
--local_ip_for_outbound_sockets=127.4.174.67
--tserver_master_addrs=127.4.174.126:40709
--webserver_port=38807
--webserver_interface=127.4.174.67
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:12:11.973906 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:11.965984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:51899 (local address 127.4.174.126:40709)
0927 07:12:11.966043 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:11.966045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:11.966284 (+ 239us) server_negotiation.cc:407] Connection header received
0927 07:12:11.966434 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:11.966435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:11.966452 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:11.966500 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:11.967202 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.967713 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.968652 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.968872 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.968926 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:11.971204 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:11.971206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:11.971206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:11.971214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:11.972490 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:11.972932 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:11.972933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:11.972934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:11.972997 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:11.973248 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:11.973249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:11.973249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:11.973371 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:11.973432 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:11.973762 (+ 330us) server_negotiation.cc:299] Negotiation successful
0927 07:12:11.973841 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 07:12:11.974131 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:11.965937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:51899)
0927 07:12:11.966176 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:11.966209 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:11.966364 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:11.966595 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:11.966599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:11.966872 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:11.967109 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:11.967115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.967814 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.967815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:11.968573 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:11.968575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.968937 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.968938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:11.969061 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:11.969820 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:11.969827 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:11.971114 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:11.972572 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:11.972580 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:11.972582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:11.972862 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:11.973056 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:11.973057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:11.973057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:11.973185 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:11.973429 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:11.973436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:11.973597 (+ 161us) client_negotiation.cc:769] Sending connection context
0927 07:12:11.973724 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 07:12:11.973900 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":87,"threads_started":1}
I20260927 07:12:11.974660 21001 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40709
I20260927 07:12:11.974756 21001 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:11.974920 21001 heartbeater.cc:511] Master 127.4.174.126:40709 requested a full tablet report, sending...
I20260927 07:12:11.975474 20673 ts_manager.cc:194] Registered new tserver with Master: 6b1af45a20d4458181512553814105da (127.4.174.66:36923)
I20260927 07:12:11.976080 20673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:51899
I20260927 07:12:11.977813 21001 heartbeater.cc:503] Master 127.4.174.126:40709 was elected leader, sending a full tablet report...
Sep 27 07:12:11 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493131, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.66@KRBTEST.COM
I20260927 07:12:11.983722 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:11.975978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55604 (local address 127.4.174.66:36923)
0927 07:12:11.976163 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:11.976168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:11.976178 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:12:11.976211 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:11.976213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:11.976227 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:11.976314 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:11.976730 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.977465 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.978390 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.978731 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.978784 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:11.981056 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:11.981064 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:11.981064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:11.981070 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:11.982513 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:11.982978 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:11.982980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:11.982981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:11.983031 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:11.983222 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:11.983223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:11.983223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:11.983345 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:11.983409 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:11.983579 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 07:12:11.983649 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260927 07:12:11.983772 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:11.975795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36923 (local address 127.0.0.1:55604)
0927 07:12:11.975856 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:11.975867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:11.975906 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:11.976318 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:11.976319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:11.976613 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:11.976650 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:11.976652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.977562 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.977564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:11.978300 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:11.978303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:11.978800 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:11.978801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:11.978861 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:11.979382 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:11.979387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:11.980968 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:11.982609 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:11.982619 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:11.982622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:11.982888 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:11.983088 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:11.983088 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:11.983089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:11.983162 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:11.983405 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:11.983414 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:11.983472 (+ 58us) client_negotiation.cc:769] Sending connection context
0927 07:12:11.983566 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 07:12:11.983673 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 07:12:11.984203 20935 tablet_service.cc:1514] Processing DeleteTablet for tablet 5485c071853b44da8a9b87d074634139 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:11:51 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:55604
I20260927 07:12:11.984761 21007 tablet_replica.cc:330] stopping tablet replica
I20260927 07:12:11.984887 21007 raft_consensus.cc:2229] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:11.984963 21007 raft_consensus.cc:2258] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:11.985306 21007 ts_tablet_manager.cc:1905] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:11.987965 21007 ts_tablet_manager.cc:1918] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:11.988054 21007 log.cc:1196] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/5485c071853b44da8a9b87d074634139
I20260927 07:12:11.988304 21007 ts_tablet_manager.cc:1939] T 5485c071853b44da8a9b87d074634139 P 6b1af45a20d4458181512553814105da: Deleting consensus metadata
I20260927 07:12:11.988999 20659 catalog_manager.cc:4841] TS 6b1af45a20d4458181512553814105da (127.4.174.66:36923): tablet 5485c071853b44da8a9b87d074634139 (table test-table [id=d77f4cd2c3dc448c9d639b307f74e0ae]) successfully deleted
W20260927 07:12:12.056356 21005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:12.056557 21005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:12.056578 21005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:12.057675 21005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:12.057709 21005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:12.057734 21005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:12:12.059119 21005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:37195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=38807
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:12.059314 21005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:12.059497 21005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:12.064167 21013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:12.064255 21015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:12.064370 21005 server_base.cc:1034] running on GCE node
W20260927 07:12:12.064407 21012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:12.064596 21005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:12.064837 21005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:12.065982 21005 hybrid_clock.cc:648] HybridClock initialized: now 1790493132065976 us; error 35 us; skew 500 ppm
Sep 27 07:12:12 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493132, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:12.068513 21005 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:12:12.068944 21005 webserver.cc:460] Webserver started at http://127.4.174.67:38807/ using document root <none> and password file <none>
I20260927 07:12:12.069139 21005 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:12.069199 21005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:12.070497 21005 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:12:12.071097 21022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:12.071244 21005 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:12.071331 21005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "d1422f8df4ad410d8efd55ff3b6b969c"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "07640f0b742989ad55c17ea0d6a752aa"
server_key_iv: "20b1f85da09c2d604783abeb06067189"
server_key_version: "encryptionkey@0"
I20260927 07:12:12.071673 21005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:12.091538 21005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:12.093699 21005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:12.093838 21005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:12.094074 21005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:12.094384 21005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:12.094417 21005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:12.094446 21005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:12.094468 21005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:12.100016 21005 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:37195
I20260927 07:12:12.100067 21135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:37195 every 8 connection(s)
I20260927 07:12:12.100415 21005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
Sep 27 07:12:12 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493132, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:12.107918 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 21005
I20260927 07:12:12.109270 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.101109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:55757 (local address 127.4.174.126:40709)
0927 07:12:12.101176 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:12.101177 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:12.101387 (+ 210us) server_negotiation.cc:407] Connection header received
0927 07:12:12.101854 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:12.101856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:12.101866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:12.101919 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:12.102883 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.103378 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.104290 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.104473 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.104520 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.106663 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:12.106664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:12.106664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:12.106672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:12.107914 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:12.108308 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:12.108309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:12.108309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:12.108368 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:12.108633 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:12.108634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:12.108635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:12.108785 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:12.108852 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:12.109086 (+ 234us) server_negotiation.cc:299] Negotiation successful
0927 07:12:12.109180 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:12:12.109364 21138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.101096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:55757)
0927 07:12:12.101271 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:12.101306 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:12.101772 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:12.101973 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:12.101977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:12.102519 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:12.102784 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:12.102789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.103443 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.103444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:12.104228 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:12.104230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.104531 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.104531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:12.104627 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.105100 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:12.105108 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:12.106540 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:12.108008 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:12.108015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:12.108017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:12.108226 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:12.108434 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:12.108435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:12.108435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:12.108560 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:12.108852 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:12.108857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:12.108971 (+ 114us) client_negotiation.cc:769] Sending connection context
0927 07:12:12.109048 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 07:12:12.109166 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260927 07:12:12.109813 21136 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40709
I20260927 07:12:12.109907 21136 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:12.110082 21136 heartbeater.cc:511] Master 127.4.174.126:40709 requested a full tablet report, sending...
I20260927 07:12:12.110452 20673 ts_manager.cc:194] Registered new tserver with Master: d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195)
I20260927 07:12:12.110931 20673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:55757
I20260927 07:12:12.114253 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.108305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49784 (local address 127.4.174.126:40709)
0927 07:12:12.108421 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:12.108423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:12.108516 (+ 93us) server_negotiation.cc:407] Connection header received
0927 07:12:12.108553 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:12.108554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:12.108566 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:12.108625 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:12.109284 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.109923 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.110892 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.111111 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.111169 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.112226 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:12.112228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:12.112228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:12.112235 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:12.113282 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:12.113566 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:12.113567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:12.113568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:12.113622 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:12.113797 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:12.113798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:12.113798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:12.113902 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:12.114056 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:12.114091 (+ 35us) server_negotiation.cc:299] Negotiation successful
0927 07:12:12.114163 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260927 07:12:12.114917 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:12:12.117223 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.115247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.4.174.126:40709)
0927 07:12:12.115294 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:12.115296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:12.115389 (+ 93us) server_negotiation.cc:407] Connection header received
0927 07:12:12.115437 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:12.115438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:12.115465 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:12.115513 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:12:12.115583 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.116018 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.116689 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.116879 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.116922 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.117041 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:12:12.117080 (+ 39us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0927 07:12:12.117184 (+ 104us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49800: token expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260927 07:12:12.117271 21139 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49800: token expired
W20260927 07:12:12.117408 20601 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.174.126:40709)
I20260927 07:12:12.122793 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.117626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.4.174.126:40709)
0927 07:12:12.117662 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:12.117664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:12.117700 (+ 36us) server_negotiation.cc:407] Connection header received
0927 07:12:12.117745 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:12.117745 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:12.117752 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:12.117787 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:12.118242 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.118734 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.119447 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.119639 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.119704 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.120735 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:12.120736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:12.120736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:12.120743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:12.121933 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:12.122193 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:12.122194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:12.122194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:12.122239 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:12.122396 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:12.122396 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:12.122397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:12.122485 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:12.122548 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:12.122664 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 07:12:12.122720 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:12:12.123855 20673 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:12:12.125427 20673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:12:12.127384 20935 tablet_service.cc:1467] Processing CreateTablet for tablet f59db4a986c1449f9a7f6f65271b307b (DEFAULT_TABLE table=test-table [id=3fa416c8d97a4b2398701edb12fb5bb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:12.127420 20795 tablet_service.cc:1467] Processing CreateTablet for tablet f59db4a986c1449f9a7f6f65271b307b (DEFAULT_TABLE table=test-table [id=3fa416c8d97a4b2398701edb12fb5bb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:12.127575 20935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f59db4a986c1449f9a7f6f65271b307b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:12.127575 20795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f59db4a986c1449f9a7f6f65271b307b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:12.129146 20753 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Bootstrap starting.
I20260927 07:12:12.129149 20892 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: Bootstrap starting.
I20260927 07:12:12.129496 20753 tablet_bootstrap.cc:654] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:12.129560 20892 tablet_bootstrap.cc:654] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:12.130059 20753 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: No bootstrap required, opened a new log
I20260927 07:12:12.130111 20753 ts_tablet_manager.cc:1397] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:12.130177 20892 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: No bootstrap required, opened a new log
I20260927 07:12:12.130228 20892 ts_tablet_manager.cc:1397] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:12.130252 20753 raft_consensus.cc:348] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.130326 20753 raft_consensus.cc:374] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:12.130386 20753 raft_consensus.cc:729] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5eab3dc2b74f6dbc160b95f86ab541, State: Initialized, Role: FOLLOWER
I20260927 07:12:12.130434 20753 consensus_queue.cc:260] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.130523 20753 ts_tablet_manager.cc:1428] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:12.130499 20892 raft_consensus.cc:348] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.130580 20892 raft_consensus.cc:374] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:12.130631 20892 raft_consensus.cc:729] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1af45a20d4458181512553814105da, State: Initialized, Role: FOLLOWER
I20260927 07:12:12.130669 20892 consensus_queue.cc:260] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.130749 20892 ts_tablet_manager.cc:1428] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:12.132512 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.127119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37195 (local address 127.0.0.1:47082)
0927 07:12:12.127160 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:12.127170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:12.127208 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:12.129485 (+ 2277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:12.129487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:12.129488 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:12.129539 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:12.129541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.130804 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.130805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:12.131657 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:12.131660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.132286 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.132287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:12.132347 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.132350 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:12.132404 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 07:12:12.132441 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260927 07:12:12.132526 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.127218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.4.174.67:37195)
0927 07:12:12.129339 (+ 2121us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:12.129343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:12.129352 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:12.129387 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:12.129388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:12.129401 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:12.129476 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:12.129602 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.130698 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.131740 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.132213 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.132273 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.132335 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:12.132419 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 07:12:12.132455 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2081,"thread_start_us":86,"threads_started":1}
I20260927 07:12:12.133248 21070 tablet_service.cc:1467] Processing CreateTablet for tablet f59db4a986c1449f9a7f6f65271b307b (DEFAULT_TABLE table=test-table [id=3fa416c8d97a4b2398701edb12fb5bb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:12.133644 21070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f59db4a986c1449f9a7f6f65271b307b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:12.134060 21004 raft_consensus.cc:482] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:12.134151 21004 raft_consensus.cc:504] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.134443 21004 leader_election.cc:288] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195), 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377)
I20260927 07:12:12.136973 21146 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: Bootstrap starting.
I20260927 07:12:12.137727 21146 tablet_bootstrap.cc:654] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:12.137879 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.134852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:47685 (local address 127.4.174.67:37195)
0927 07:12:12.134887 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:12.134889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:12.134895 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:12:12.134914 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:12.134915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:12.134922 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:12.134962 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:12.135133 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.135825 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.137019 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.137506 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.137571 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.137579 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:12.137778 (+ 199us) server_negotiation.cc:299] Negotiation successful
0927 07:12:12.137817 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:12:12.137885 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.134627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37195 (local address 127.4.174.66:47685)
0927 07:12:12.134714 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:12.134724 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:12.134773 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:12.135009 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:12.135010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:12.135011 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:12.135067 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:12.135068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.135924 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.135925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:12.136960 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:12.136961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.137612 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.137613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:12.137687 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.137712 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 07:12:12.137772 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:12:12.137818 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260927 07:12:12.137975 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.134688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:34415 (local address 127.4.174.65:37377)
0927 07:12:12.134729 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:12.134730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:12.134868 (+ 138us) server_negotiation.cc:407] Connection header received
0927 07:12:12.134921 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:12.134922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:12.134930 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:12.134974 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:12.135094 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.135592 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.136948 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.137390 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.137428 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.137435 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:12.137868 (+ 433us) server_negotiation.cc:299] Negotiation successful
0927 07:12:12.137918 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:12:12.138203 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.134641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:37377 (local address 127.4.174.66:34415)
0927 07:12:12.134810 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:12.134818 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:12.134866 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:12.134975 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:12.134977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:12.134978 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:12.135038 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:12.135040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.135658 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.135659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:12.136847 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:12.136850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.137701 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.137703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:12.137784 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.137788 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:12:12.138072 (+ 284us) client_negotiation.cc:240] Negotiation successful
0927 07:12:12.138118 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
I20260927 07:12:12.138443 21146 log.cc:824] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:12.138459 20815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59db4a986c1449f9a7f6f65271b307b" candidate_uuid: "6b1af45a20d4458181512553814105da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" is_pre_election: true
I20260927 07:12:12.138618 20815 raft_consensus.cc:2454] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b1af45a20d4458181512553814105da in term 0.
I20260927 07:12:12.138167 21090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59db4a986c1449f9a7f6f65271b307b" candidate_uuid: "6b1af45a20d4458181512553814105da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" is_pre_election: true
I20260927 07:12:12.139014 20888 leader_election.cc:302] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541, 6b1af45a20d4458181512553814105da; no voters:
I20260927 07:12:12.139140 21004 raft_consensus.cc:2781] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:12:12.139189 21004 raft_consensus.cc:482] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:12:12.139213 21004 raft_consensus.cc:3037] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 0 FOLLOWER]: Advancing to term 1
W20260927 07:12:12.139344 20887 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): Illegal state: must be running to vote when last-logged opid is not known
I20260927 07:12:12.139559 21146 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: No bootstrap required, opened a new log
I20260927 07:12:12.139624 21146 ts_tablet_manager.cc:1397] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:12:12.139937 21004 raft_consensus.cc:504] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.140056 21004 leader_election.cc:288] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 election: Requested vote from peers d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195), 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377)
I20260927 07:12:12.140208 20815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59db4a986c1449f9a7f6f65271b307b" candidate_uuid: "6b1af45a20d4458181512553814105da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541"
I20260927 07:12:12.140275 21090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59db4a986c1449f9a7f6f65271b307b" candidate_uuid: "6b1af45a20d4458181512553814105da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1422f8df4ad410d8efd55ff3b6b969c"
I20260927 07:12:12.140298 20815 raft_consensus.cc:3037] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 0 FOLLOWER]: Advancing to term 1
W20260927 07:12:12.140424 20887 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): Illegal state: must be running to vote when last-logged opid is not known
I20260927 07:12:12.140776 21146 raft_consensus.cc:348] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.140908 21146 raft_consensus.cc:374] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:12.140941 21146 raft_consensus.cc:729] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1422f8df4ad410d8efd55ff3b6b969c, State: Initialized, Role: FOLLOWER
I20260927 07:12:12.141057 21146 consensus_queue.cc:260] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.141306 21146 ts_tablet_manager.cc:1428] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:12:12.141383 21136 heartbeater.cc:503] Master 127.4.174.126:40709 was elected leader, sending a full tablet report...
I20260927 07:12:12.141568 20815 raft_consensus.cc:2454] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b1af45a20d4458181512553814105da in term 1.
I20260927 07:12:12.141726 20888 leader_election.cc:302] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541, 6b1af45a20d4458181512553814105da; no voters: d1422f8df4ad410d8efd55ff3b6b969c
I20260927 07:12:12.141819 21004 raft_consensus.cc:2781] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:12.141961 21004 raft_consensus.cc:686] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 1 LEADER]: Becoming Leader. State: Replica: 6b1af45a20d4458181512553814105da, State: Running, Role: LEADER
I20260927 07:12:12.142035 21004 consensus_queue.cc:237] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:12.142735 20673 catalog_manager.cc:5495] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b1af45a20d4458181512553814105da (127.4.174.66). New cstate: current_term: 1 leader_uuid: "6b1af45a20d4458181512553814105da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } health_report { overall_health: HEALTHY } } }
I20260927 07:12:12.148900 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.146815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.4.174.66:36923)
0927 07:12:12.146861 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:12.146863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:12.146907 (+ 44us) server_negotiation.cc:407] Connection header received
0927 07:12:12.146956 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:12.146957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:12.146972 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:12.147027 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:12:12.147112 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.147581 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.148318 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:12.148508 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:12.148556 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:12.148664 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:12:12.148758 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:12:12.148820 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:12.148833 (+ 13us) server_negotiation.cc:299] Negotiation successful
0927 07:12:12.148849 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 07:12:12.150153 21090 raft_consensus.cc:3037] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:12.150168 20815 raft_consensus.cc:1261] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Refusing update from remote peer 6b1af45a20d4458181512553814105da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:12.150475 21149 consensus_queue.cc:1035] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:12.150976 21090 raft_consensus.cc:1261] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c [term 1 FOLLOWER]: Refusing update from remote peer 6b1af45a20d4458181512553814105da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:12.151214 21149 consensus_queue.cc:1035] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:12.152503 21152 mvcc.cc:205] Tried to move back new op lower bound from 7333859869285089280 to 7333859869254389760. Current Snapshot: MvccSnapshot[applied={T|T < 7333859869285089280}]
I20260927 07:12:12.154227 21155 mvcc.cc:205] Tried to move back new op lower bound from 7333859869285089280 to 7333859869254389760. Current Snapshot: MvccSnapshot[applied={T|T < 7333859869285089280}]
I20260927 07:12:12.154549 20673 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:12.154657 20673 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:12.155321 20673 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Sending DeleteTablet for 3 replicas of tablet f59db4a986c1449f9a7f6f65271b307b
I20260927 07:12:12.155598 20795 tablet_service.cc:1514] Processing DeleteTablet for tablet f59db4a986c1449f9a7f6f65271b307b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:12 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:35056
I20260927 07:12:12.155616 20935 tablet_service.cc:1514] Processing DeleteTablet for tablet f59db4a986c1449f9a7f6f65271b307b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:12 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:55604
I20260927 07:12:12.155711 21007 tablet_replica.cc:330] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: stopping tablet replica
I20260927 07:12:12.155750 21070 tablet_service.cc:1514] Processing DeleteTablet for tablet f59db4a986c1449f9a7f6f65271b307b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:12 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:47082
I20260927 07:12:12.155781 21007 raft_consensus.cc:2229] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:12:12.155865 21007 raft_consensus.cc:2258] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:12.155941 20870 tablet_replica.cc:330] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: stopping tablet replica
I20260927 07:12:12.155983 21161 tablet_replica.cc:330] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: stopping tablet replica
I20260927 07:12:12.156001 20870 raft_consensus.cc:2229] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:12.156059 20870 raft_consensus.cc:2258] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:12.156082 21161 raft_consensus.cc:2229] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:12.156225 21007 ts_tablet_manager.cc:1905] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:12.156247 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 20711
I20260927 07:12:12.157711 21007 ts_tablet_manager.cc:1918] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:12.157773 21007 log.cc:1196] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/f59db4a986c1449f9a7f6f65271b307b
I20260927 07:12:12.157975 21007 ts_tablet_manager.cc:1939] T f59db4a986c1449f9a7f6f65271b307b P 6b1af45a20d4458181512553814105da: Deleting consensus metadata
I20260927 07:12:12.158300 20659 catalog_manager.cc:4841] TS 6b1af45a20d4458181512553814105da (127.4.174.66:36923): tablet f59db4a986c1449f9a7f6f65271b307b (table test-table [id=3fa416c8d97a4b2398701edb12fb5bb4]) successfully deleted
I20260927 07:12:12.156278 21161 raft_consensus.cc:2258] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:12.158911 21161 ts_tablet_manager.cc:1905] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:12.160442 21161 ts_tablet_manager.cc:1918] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:12.160511 21161 log.cc:1196] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/f59db4a986c1449f9a7f6f65271b307b
W20260927 07:12:12.160717 20659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:37377 (error 108)
I20260927 07:12:12.160785 21161 ts_tablet_manager.cc:1939] T f59db4a986c1449f9a7f6f65271b307b P d1422f8df4ad410d8efd55ff3b6b969c: Deleting consensus metadata
I20260927 07:12:12.161020 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 20866
I20260927 07:12:12.161201 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:12.161072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:37377 (local address 127.0.0.1:35072)
0927 07:12:12.161119 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:12.161170 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:37377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 07:12:12.161322 20658 catalog_manager.cc:4841] TS d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): tablet f59db4a986c1449f9a7f6f65271b307b (table test-table [id=3fa416c8d97a4b2398701edb12fb5bb4]) successfully deleted
W20260927 07:12:12.162859 20659 catalog_manager.cc:4584] TS 2d5eab3dc2b74f6dbc160b95f86ab541 (127.4.174.65:37377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f59db4a986c1449f9a7f6f65271b307b: Network error: Client connection negotiation failed: client connection to 127.4.174.65:37377: connect: Connection refused (error 111)
I20260927 07:12:12.165329 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 21005
I20260927 07:12:12.169592 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 20640
I20260927 07:12:32.174427 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40709
--webserver_interface=127.4.174.126
--webserver_port=41175
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:32.251966 21163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:32.252161 21163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:32.252182 21163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:32.253321 21163 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 07:12:32.253360 21163 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 07:12:32.253371 21163 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 07:12:32.253381 21163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:32.253392 21163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:32.253403 21163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:32.253415 21163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:32.253429 21163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:32.254917 21163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:32.255133 21163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:32.255345 21163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:32.260160 21171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:32.260259 21163 server_base.cc:1034] running on GCE node
W20260927 07:12:32.260241 21168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:32.260241 21169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:32.260685 21163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:32.260960 21163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:32.262112 21163 hybrid_clock.cc:648] HybridClock initialized: now 1790493152262099 us; error 30 us; skew 500 ppm
Sep 27 07:12:32 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493152, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:32.269910 21163 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:12:32.270329 21163 webserver.cc:460] Webserver started at http://127.4.174.126:41175/ using document root <none> and password file <none>
I20260927 07:12:32.270542 21163 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:32.270622 21163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:32.271914 21163 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:32.272516 21178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.272667 21163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.272750 21163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "574ddf87171547f4a31780fd258bfc96"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "1ba5b39104163592bb10c9a2a3aa49db"
server_key_iv: "53754509475ff66c6dd43361a3ce7550"
server_key_version: "encryptionkey@0"
I20260927 07:12:32.273075 21163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:32.296599 21163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:32.298368 21163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:32.298502 21163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:32.302137 21163 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40709
I20260927 07:12:32.302177 21230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40709 every 8 connection(s)
I20260927 07:12:32.302531 21163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:32.303973 21231 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:12:32.304613 21231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Bootstrap starting.
I20260927 07:12:32.309290 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 21163
I20260927 07:12:32.309546 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:37377
--local_ip_for_outbound_sockets=127.4.174.65
--tserver_master_addrs=127.4.174.126:40709
--webserver_port=38409
--webserver_interface=127.4.174.65
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:12:32.312453 21231 log.cc:824] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:32.314512 21231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:12:32.314796 21231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Bootstrap complete.
I20260927 07:12:32.316850 21231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:12:32.317062 21231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 574ddf87171547f4a31780fd258bfc96, State: Initialized, Role: FOLLOWER
I20260927 07:12:32.317204 21231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:12:32.317286 21231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:32.317334 21231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:32.317379 21231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 2 FOLLOWER]: Advancing to term 3
I20260927 07:12:32.318306 21231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:12:32.318394 21231 leader_election.cc:302] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 574ddf87171547f4a31780fd258bfc96; no voters:
I20260927 07:12:32.318570 21231 leader_election.cc:288] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 07:12:32.318615 21235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 07:12:32.318735 21235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [term 3 LEADER]: Becoming Leader. State: Replica: 574ddf87171547f4a31780fd258bfc96, State: Running, Role: LEADER
I20260927 07:12:32.318872 21231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:32.318887 21235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } }
I20260927 07:12:32.319304 21237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 574ddf87171547f4a31780fd258bfc96. Latest consensus state: current_term: 3 leader_uuid: "574ddf87171547f4a31780fd258bfc96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } } }
I20260927 07:12:32.319316 21236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "574ddf87171547f4a31780fd258bfc96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "574ddf87171547f4a31780fd258bfc96" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40709 } } }
I20260927 07:12:32.319394 21237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:32.319422 21236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:32.319885 21243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:32.320693 21243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:32.320859 21243 catalog_manager.cc:1259] Loaded cluster ID: de2331e82dd241ada3858f293ca484c3
I20260927 07:12:32.321033 21243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:32.321882 21243 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:32.322039 21243 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Loaded TSK: 0
I20260927 07:12:32.322073 21243 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 574ddf87171547f4a31780fd258bfc96: Loaded TSK: 1
F20260927 07:12:32.322286 21243 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f20202769ad google::LogMessage::Fail() at ??:0
@ 0x7f202027988a google::LogMessage::SendToLog() at ??:0
@ 0x7f20202764de google::LogMessage::Flush() at ??:0
@ 0x7f2020277a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f20229e3104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f20229e0d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f20281d01cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f20281d048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f20281d0594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f2028196c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f20281a0ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f20221440d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2022139954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2022f9d6db start_thread at ??:0
@ 0x7f201ed7461f clone at ??:0
W20260927 07:12:32.408067 21233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:32.408257 21233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:32.408277 21233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:32.409408 21233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:32.409451 21233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:32.409475 21233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:12:32.410952 21233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:37377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=38409
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:32.411270 21233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:32.411495 21233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:32.417167 21272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:32.417162 21273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:32.417167 21275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:32.417757 21233 server_base.cc:1034] running on GCE node
I20260927 07:12:32.417938 21233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:32.418166 21233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:32.419337 21233 hybrid_clock.cc:648] HybridClock initialized: now 1790493152419320 us; error 32 us; skew 500 ppm
Sep 27 07:12:32 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493152, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:32.421878 21233 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:12:32.422312 21233 webserver.cc:460] Webserver started at http://127.4.174.65:38409/ using document root <none> and password file <none>
I20260927 07:12:32.422528 21233 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:32.422592 21233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:32.423875 21233 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:12:32.424528 21282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.424695 21233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:12:32.424773 21233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "2d5eab3dc2b74f6dbc160b95f86ab541"
format_stamp: "Formatted at 2026-09-27 07:11:50 on dist-test-slave-zw7p"
server_key: "688e36ecbfd948094fda3a1e4ac6c00f"
server_key_iv: "f4361d508710c22b03d6e459a1b1df3a"
server_key_version: "encryptionkey@0"
I20260927 07:12:32.425104 21233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:32.438985 21233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:32.441051 21233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:32.441184 21233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:32.441390 21233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:32.441807 21289 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 07:12:32.443765 21233 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 07:12:32.443817 21233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 07:12:32.443852 21233 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 07:12:32.444478 21233 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 07:12:32.444509 21233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 07:12:32.444609 21289 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Bootstrap starting.
I20260927 07:12:32.450917 21233 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:37377
I20260927 07:12:32.451285 21233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:12:32.454484 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 21233
I20260927 07:12:32.454734 21396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:37377 every 8 connection(s)
I20260927 07:12:32.454802 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:36923
--local_ip_for_outbound_sockets=127.4.174.66
--tserver_master_addrs=127.4.174.126:40709
--webserver_port=35461
--webserver_interface=127.4.174.66
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:12:32.454885 21289 log.cc:824] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:32.457528 21289 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:12:32.457815 21289 tablet_bootstrap.cc:492] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Bootstrap complete.
I20260927 07:12:32.458854 21289 ts_tablet_manager.cc:1397] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.007s
I20260927 07:12:32.460178 21289 raft_consensus.cc:348] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:32.460502 21289 raft_consensus.cc:729] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5eab3dc2b74f6dbc160b95f86ab541, State: Initialized, Role: FOLLOWER
I20260927 07:12:32.460639 21289 consensus_queue.cc:260] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:32.460892 21289 ts_tablet_manager.cc:1428] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260927 07:12:32.497718 21399 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:12:32.451894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:41443)
0927 07:12:32.452051 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.452093 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:32.452533 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:32.497509 (+ 44976us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
W20260927 07:12:32.497987 21397 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40709 (0 consecutive failures): Network error: Failed to ping master at 127.4.174.126:40709: Client connection negotiation failed: client connection to 127.4.174.126:40709: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 07:12:32.498301 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:32.498194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:47095)
0927 07:12:32.498241 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.498281 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 07:12:32.498641 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:32.498527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:34645)
0927 07:12:32.498590 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.498617 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260927 07:12:32.498749 21397 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:12:32.537246 21401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:32.537463 21401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:32.537487 21401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:32.538636 21401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:32.538677 21401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:32.538703 21401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:12:32.540141 21401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:36923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=35461
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:32.540333 21401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:32.540521 21401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:32.545455 21407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:32.545469 21410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:32.545469 21408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:32.545718 21401 server_base.cc:1034] running on GCE node
I20260927 07:12:32.545928 21401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:32.546155 21401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:32.547294 21401 hybrid_clock.cc:648] HybridClock initialized: now 1790493152547283 us; error 28 us; skew 500 ppm
Sep 27 07:12:32 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493152, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:32.549613 21401 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:12:32.550083 21401 webserver.cc:460] Webserver started at http://127.4.174.66:35461/ using document root <none> and password file <none>
I20260927 07:12:32.550289 21401 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:32.550336 21401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:32.551651 21401 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:32.552196 21417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.552343 21401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:32.552420 21401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "6b1af45a20d4458181512553814105da"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "b4dab5ee2313f6dc768298585f3460d9"
server_key_iv: "2cba961a7cec05090df21a5256e9f6ce"
server_key_version: "encryptionkey@0"
I20260927 07:12:32.552778 21401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:32.560580 21401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:32.562642 21401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:32.562776 21401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:32.562980 21401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:32.563274 21401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:32.563306 21401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.563334 21401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:32.563357 21401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.568917 21401 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:36923
I20260927 07:12:32.569002 21530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:36923 every 8 connection(s)
I20260927 07:12:32.569305 21401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:12:32.570379 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:32.569942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:55587)
0927 07:12:32.570168 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.570242 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":85,"threads_started":1}
W20260927 07:12:32.570793 21531 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40709 (0 consecutive failures): Network error: Failed to ping master at 127.4.174.126:40709: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
I20260927 07:12:32.571058 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:32.570968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:48509)
0927 07:12:32.571007 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.571036 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260927 07:12:32.571282 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:32.571220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:42089)
0927 07:12:32.571249 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.571269 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260927 07:12:32.571372 21531 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:12:32.571759 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 21401
I20260927 07:12:32.572014 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:37195
--local_ip_for_outbound_sockets=127.4.174.67
--tserver_master_addrs=127.4.174.126:40709
--webserver_port=38807
--webserver_interface=127.4.174.67
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:32.651069 21534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:32.651279 21534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:32.651302 21534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:32.652451 21534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:32.652493 21534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:32.652518 21534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:12:32.653947 21534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:37195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=38807
--tserver_master_addrs=127.4.174.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:32.654171 21534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:32.654390 21534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:32.659192 21540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:32.659250 21539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:32.659283 21542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:32.659420 21534 server_base.cc:1034] running on GCE node
I20260927 07:12:32.659617 21534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:32.659823 21534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:32.660964 21534 hybrid_clock.cc:648] HybridClock initialized: now 1790493152660948 us; error 30 us; skew 500 ppm
Sep 27 07:12:32 dist-test-slave-zw7p krb5kdc[20091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493152, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:32.663511 21534 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:12:32.663961 21534 webserver.cc:460] Webserver started at http://127.4.174.67:38807/ using document root <none> and password file <none>
I20260927 07:12:32.664161 21534 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:32.664207 21534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:32.665640 21534 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:12:32.666316 21549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.666498 21534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:12:32.666617 21534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "d1422f8df4ad410d8efd55ff3b6b969c"
format_stamp: "Formatted at 2026-09-27 07:11:51 on dist-test-slave-zw7p"
server_key: "07640f0b742989ad55c17ea0d6a752aa"
server_key_iv: "20b1f85da09c2d604783abeb06067189"
server_key_version: "encryptionkey@0"
I20260927 07:12:32.667012 21534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:32.676409 21534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:32.678356 21534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:32.678498 21534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:32.678766 21534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:32.679052 21534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:32.679085 21534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.679112 21534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:32.679136 21534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:32.684602 21534 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:37195
I20260927 07:12:32.684659 21662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:37195 every 8 connection(s)
I20260927 07:12:32.684984 21534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:12:32.686028 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:32.685622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:48051)
0927 07:12:32.685817 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.685888 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
W20260927 07:12:32.686446 21663 heartbeater.cc:644] Failed to heartbeat to 127.4.174.126:40709 (0 consecutive failures): Network error: Failed to ping master at 127.4.174.126:40709: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
I20260927 07:12:32.686743 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 21534
I20260927 07:12:32.686780 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:32.686676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:44351)
0927 07:12:32.686724 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.686757 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 07:12:32.687108 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:32.687018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:58657)
0927 07:12:32.687062 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:32.687088 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 07:12:32.687197 21663 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:12:33.499300 21667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:33.499084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:58373)
0927 07:12:33.499200 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:33.499237 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260927 07:12:33.571905 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:33.571713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:37377)
0927 07:12:33.571818 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:33.571854 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
I20260927 07:12:33.687681 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:33.687485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:51857)
0927 07:12:33.687599 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:33.687633 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
I20260927 07:12:33.857734 21670 raft_consensus.cc:482] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:33.857859 21670 raft_consensus.cc:504] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:33.858181 21670 leader_election.cc:288] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195), 6b1af45a20d4458181512553814105da (127.4.174.66:36923)
Sep 27 07:12:33 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493152, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.67@KRBTEST.COM
Sep 27 07:12:33 dist-test-slave-zw7p krb5kdc[20091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493152, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.66@KRBTEST.COM
I20260927 07:12:33.868216 21667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:33.858328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37195 (local address 127.4.174.65:37043)
0927 07:12:33.858378 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:33.858391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:33.858452 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:33.859305 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:33.859308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:33.859664 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:33.859847 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:33.859852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:33.860863 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:33.860864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:33.861608 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:33.861610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:33.862145 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:33.862146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:33.862250 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:33.862771 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:33.862779 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:33.864323 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:33.866703 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:33.866710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:33.866712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:33.866990 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:33.867240 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:33.867241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:33.867241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:33.867349 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:33.867764 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:33.867770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:33.867903 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 07:12:33.867979 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 07:12:33.868118 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":49}
I20260927 07:12:33.868635 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:33.858444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:37043 (local address 127.4.174.67:37195)
0927 07:12:33.858603 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:33.858614 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:33.858640 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:12:33.859076 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:33.859078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:33.859123 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:33.859246 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:33.859950 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:33.860772 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:33.861684 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:33.862064 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:33.862156 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:33.864451 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:33.864455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:33.864456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:33.864469 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:33.866628 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:33.867074 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:33.867077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:33.867077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:33.867176 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:33.867425 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:33.867425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:33.867426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:33.867697 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:33.868331 (+ 634us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:33.868415 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 07:12:33.868535 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
W20260927 07:12:33.871532 21284 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
I20260927 07:12:33.872795 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:33.858340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36923 (local address 127.4.174.65:40581)
0927 07:12:33.858477 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:33.858486 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:33.858524 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:33.859304 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:33.859308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:33.859666 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:33.859847 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:33.859852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:33.860880 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:33.860880 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:33.861606 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:33.861608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:33.862142 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:33.862143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:33.862245 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:33.862810 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:33.862814 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:33.864727 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:33.866735 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:33.866735 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:33.866736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:33.871696 (+ 4960us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:33.871964 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:33.871980 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:33.871981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:33.872044 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:33.872486 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:33.872487 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:33.872532 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 07:12:33.872594 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 07:12:33.872691 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":6,"thread_start_us":82,"threads_started":1}
I20260927 07:12:33.872879 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:33.858470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:40581 (local address 127.4.174.66:36923)
0927 07:12:33.858634 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:33.858642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:33.858670 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:12:33.859084 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:33.859086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:33.859123 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:33.859248 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:33.859964 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:33.860784 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:33.861682 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:33.862064 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:33.862156 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:33.864837 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:33.864840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:33.864840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:33.864851 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:33.866676 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:33.871796 (+ 5120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:33.871800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:33.871800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:33.871905 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:33.872103 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:33.872104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:33.872105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:33.872421 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:33.872496 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:33.872659 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 07:12:33.872785 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":41,"threads_started":1}
W20260927 07:12:33.873492 21285 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b1af45a20d4458181512553814105da (127.4.174.66:36923): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
I20260927 07:12:33.873566 21285 leader_election.cc:302] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541; no voters: 6b1af45a20d4458181512553814105da, d1422f8df4ad410d8efd55ff3b6b969c
I20260927 07:12:33.873688 21670 raft_consensus.cc:2726] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:12:34.499940 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:34.499710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:55587)
0927 07:12:34.499832 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:34.499877 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260927 07:12:34.572631 21675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:34.572401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:45827)
0927 07:12:34.572543 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:34.572586 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":60,"threads_started":1}
I20260927 07:12:34.688262 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:34.688061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:45801)
0927 07:12:34.688179 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:34.688221 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":55,"threads_started":1}
I20260927 07:12:35.500653 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:35.500428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:51905)
0927 07:12:35.500570 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:35.500608 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260927 07:12:35.573314 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:35.573095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:41115)
0927 07:12:35.573227 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:35.573263 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260927 07:12:35.688850 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:35.688652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:44639)
0927 07:12:35.688773 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:35.688809 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260927 07:12:35.967013 21680 raft_consensus.cc:482] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:35.967103 21680 raft_consensus.cc:504] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:35.967252 21680 leader_election.cc:288] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195), 6b1af45a20d4458181512553814105da (127.4.174.66:36923)
W20260927 07:12:35.967648 21285 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b1af45a20d4458181512553814105da (127.4.174.66:36923): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
W20260927 07:12:35.967761 21284 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
I20260927 07:12:35.967829 21284 leader_election.cc:302] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541; no voters: 6b1af45a20d4458181512553814105da, d1422f8df4ad410d8efd55ff3b6b969c
I20260927 07:12:35.967897 21680 raft_consensus.cc:2726] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:12:36.501370 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:36.501141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:48537)
0927 07:12:36.501283 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:36.501323 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260927 07:12:36.574012 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:36.573789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:59011)
0927 07:12:36.573932 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:36.573972 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260927 07:12:36.689488 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:36.689272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:51049)
0927 07:12:36.689406 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:36.689447 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
I20260927 07:12:37.502126 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:37.501916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:58583)
0927 07:12:37.502042 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:37.502080 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260927 07:12:37.574807 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:37.574587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:52341)
0927 07:12:37.574718 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:37.574762 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260927 07:12:37.690155 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:37.689914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:51161)
0927 07:12:37.690058 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:37.690108 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":60,"threads_started":1}
I20260927 07:12:37.756093 21687 raft_consensus.cc:482] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:37.756196 21687 raft_consensus.cc:504] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:37.756352 21687 leader_election.cc:288] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195), 6b1af45a20d4458181512553814105da (127.4.174.66:36923)
W20260927 07:12:37.756649 21285 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b1af45a20d4458181512553814105da (127.4.174.66:36923): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
W20260927 07:12:37.756737 21284 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
I20260927 07:12:37.756781 21284 leader_election.cc:302] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541; no voters: 6b1af45a20d4458181512553814105da, d1422f8df4ad410d8efd55ff3b6b969c
I20260927 07:12:37.756844 21687 raft_consensus.cc:2726] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:12:38.502756 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:38.502523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:51395)
0927 07:12:38.502663 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:38.502699 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260927 07:12:38.575546 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:38.575312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:50447)
0927 07:12:38.575445 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:38.575497 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260927 07:12:38.690814 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:38.690592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:42953)
0927 07:12:38.690727 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:38.690761 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260927 07:12:39.503372 21691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:39.503152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:39359)
0927 07:12:39.503288 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:39.503324 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260927 07:12:39.576275 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:39.576058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:60465)
0927 07:12:39.576187 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:39.576229 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260927 07:12:39.691468 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:39.691245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:56583)
0927 07:12:39.691385 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:39.691424 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260927 07:12:40.503998 21694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:40.503798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:45041)
0927 07:12:40.503919 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:40.503952 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260927 07:12:40.576926 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:40.576715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:48851)
0927 07:12:40.576849 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:40.576884 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260927 07:12:40.692104 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:40.691883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:41317)
0927 07:12:40.692015 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:40.692062 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260927 07:12:41.397430 21697 raft_consensus.cc:482] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:41.397526 21697 raft_consensus.cc:504] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:41.397710 21697 leader_election.cc:288] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195), 6b1af45a20d4458181512553814105da (127.4.174.66:36923)
W20260927 07:12:41.398041 21284 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
W20260927 07:12:41.398128 21285 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b1af45a20d4458181512553814105da (127.4.174.66:36923): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
I20260927 07:12:41.398187 21285 leader_election.cc:302] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541; no voters: 6b1af45a20d4458181512553814105da, d1422f8df4ad410d8efd55ff3b6b969c
I20260927 07:12:41.398264 21697 raft_consensus.cc:2726] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:12:41.504675 21698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:41.504459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:55667)
0927 07:12:41.504579 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:41.504622 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260927 07:12:41.577630 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:41.577415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:36251)
0927 07:12:41.577528 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:41.577569 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260927 07:12:41.692813 21700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:41.692595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:48873)
0927 07:12:41.692727 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:41.692767 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260927 07:12:42.505414 21701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:42.505174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:42493)
0927 07:12:42.505319 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:42.505363 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
I20260927 07:12:42.578327 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:42.578105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:46923)
0927 07:12:42.578238 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:42.578280 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260927 07:12:42.693539 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:42.693306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:45559)
0927 07:12:42.693445 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:42.693491 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
I20260927 07:12:43.506145 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:43.505903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:36727)
0927 07:12:43.506054 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:43.506097 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260927 07:12:43.579033 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:43.578807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:43405)
0927 07:12:43.578928 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:43.578971 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260927 07:12:43.694301 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:43.694084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:57673)
0927 07:12:43.694210 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:43.694254 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260927 07:12:44.506887 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:44.506685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:45857)
0927 07:12:44.506798 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:44.506840 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260927 07:12:44.579770 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:44.579525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:41725)
0927 07:12:44.579673 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:44.579719 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260927 07:12:44.694939 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:44.694737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:54251)
0927 07:12:44.694853 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:44.694894 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260927 07:12:45.507476 21710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:45.507280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:44497)
0927 07:12:45.507394 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:45.507429 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260927 07:12:45.580505 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:45.580239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:37123)
0927 07:12:45.580395 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:45.580434 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":58,"threads_started":1}
I20260927 07:12:45.695618 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:45.695384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:33385)
0927 07:12:45.695538 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:45.695574 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":61,"threads_started":1}
I20260927 07:12:46.508134 21713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:46.507902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:54957)
0927 07:12:46.508040 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:46.508079 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260927 07:12:46.581251 21714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:46.580976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:53249)
0927 07:12:46.581147 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:46.581198 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":65,"threads_started":1}
I20260927 07:12:46.696273 21715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:46.696047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:45605)
0927 07:12:46.696178 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:46.696218 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260927 07:12:47.508999 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:47.508766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.65:52037)
0927 07:12:47.508891 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:47.508943 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260927 07:12:47.569800 21717 raft_consensus.cc:482] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:47.569892 21717 raft_consensus.cc:504] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1422f8df4ad410d8efd55ff3b6b969c" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37195 } } peers { permanent_uuid: "2d5eab3dc2b74f6dbc160b95f86ab541" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 37377 } } peers { permanent_uuid: "6b1af45a20d4458181512553814105da" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36923 } }
I20260927 07:12:47.570042 21717 leader_election.cc:288] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195), 6b1af45a20d4458181512553814105da (127.4.174.66:36923)
W20260927 07:12:47.570408 21284 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1422f8df4ad410d8efd55ff3b6b969c (127.4.174.67:37195): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
W20260927 07:12:47.570492 21285 leader_election.cc:341] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b1af45a20d4458181512553814105da (127.4.174.66:36923): Not found: Tablet not found: f59db4a986c1449f9a7f6f65271b307b
I20260927 07:12:47.570568 21285 leader_election.cc:302] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d5eab3dc2b74f6dbc160b95f86ab541; no voters: 6b1af45a20d4458181512553814105da, d1422f8df4ad410d8efd55ff3b6b969c
I20260927 07:12:47.570657 21717 raft_consensus.cc:2726] T f59db4a986c1449f9a7f6f65271b307b P 2d5eab3dc2b74f6dbc160b95f86ab541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:12:47.582064 21718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:47.581851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.66:35703)
0927 07:12:47.581979 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:47.582021 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260927 07:12:47.692119 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 21233
I20260927 07:12:47.696463 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 21401
I20260927 07:12:47.697219 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:47.696705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40709 (local address 127.4.174.67:38259)
0927 07:12:47.697141 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:47.697174 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.126:40709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":48,"threads_started":1}
I20260927 07:12:47.700541 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 21534
2026-09-27T07:12:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58139 ms)
[----------] 30 tests from SecurityITest (226710 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:12:47 dist-test-slave-zw7p krb5kdc[21722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:12:47 dist-test-slave-zw7p krb5kdc[21722](info): set up 2 sockets
Sep 27 07:12:47 dist-test-slave-zw7p krb5kdc[21722](info): commencing operation
krb5kdc: starting...
W20260927 07:12:48.646026 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:12:48 dist-test-slave-zw7p krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:12:48.757949 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:41825
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:41825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:48.835103 21735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:48.835311 21735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:48.835341 21735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:48.836519 21735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:48.836561 21735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:48.836580 21735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:48.836596 21735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:48.836611 21735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:48.838068 21735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:41825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:41825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:48.838306 21735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:48.838510 21735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:48.843295 21740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:48.843312 21743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:48.843390 21735 server_base.cc:1034] running on GCE node
W20260927 07:12:48.843398 21741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:48.843775 21735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:48.844017 21735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:48.845152 21735 hybrid_clock.cc:648] HybridClock initialized: now 1790493168845139 us; error 29 us; skew 500 ppm
Sep 27 07:12:48 dist-test-slave-zw7p krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:48.847401 21735 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:12:48.847805 21735 webserver.cc:460] Webserver started at http://127.4.174.126:43569/ using document root <none> and password file <none>
I20260927 07:12:48.848014 21735 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:48.848061 21735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:48.848181 21735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:48.849190 21735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "0bb88e1a5c444928ace50b82b345e7e3"
format_stamp: "Formatted at 2026-09-27 07:12:48 on dist-test-slave-zw7p"
server_key: "9f7140963c72d6a4fca38a3ea92ee1e4"
server_key_iv: "033f686b6abb8513a90625bdb8ef8ea4"
server_key_version: "encryptionkey@0"
I20260927 07:12:48.849504 21735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "0bb88e1a5c444928ace50b82b345e7e3"
format_stamp: "Formatted at 2026-09-27 07:12:48 on dist-test-slave-zw7p"
server_key: "9f7140963c72d6a4fca38a3ea92ee1e4"
server_key_iv: "033f686b6abb8513a90625bdb8ef8ea4"
server_key_version: "encryptionkey@0"
I20260927 07:12:48.850759 21735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:12:48.851426 21750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:48.851565 21735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:48.851646 21735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "0bb88e1a5c444928ace50b82b345e7e3"
format_stamp: "Formatted at 2026-09-27 07:12:48 on dist-test-slave-zw7p"
server_key: "9f7140963c72d6a4fca38a3ea92ee1e4"
server_key_iv: "033f686b6abb8513a90625bdb8ef8ea4"
server_key_version: "encryptionkey@0"
I20260927 07:12:48.851702 21735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:48.865896 21735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:48.867796 21735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:48.867921 21735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:48.871376 21735 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:41825
I20260927 07:12:48.871419 21802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:41825 every 8 connection(s)
I20260927 07:12:48.871728 21735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:48.872152 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 21735
I20260927 07:12:48.872262 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:48.872452 4793 external_mini_cluster.cc:1428] Setting key b55b6abc1658fc8ed689a0148304cbce
I20260927 07:12:48.872771 21803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:48.875530 21803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3: Bootstrap starting.
I20260927 07:12:48.876067 21803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:48.876331 21803 log.cc:824] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:48.876932 21803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3: No bootstrap required, opened a new log
Sep 27 07:12:48 dist-test-slave-zw7p krb5kdc[21722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:48.878866 21803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb88e1a5c444928ace50b82b345e7e3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41825 } }
I20260927 07:12:48.879000 21803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:48.879022 21803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bb88e1a5c444928ace50b82b345e7e3, State: Initialized, Role: FOLLOWER
I20260927 07:12:48.879133 21803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb88e1a5c444928ace50b82b345e7e3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41825 } }
I20260927 07:12:48.879196 21803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:48.879216 21803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:48.879249 21803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:48.879843 21803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb88e1a5c444928ace50b82b345e7e3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41825 } }
I20260927 07:12:48.879952 21803 leader_election.cc:302] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bb88e1a5c444928ace50b82b345e7e3; no voters:
I20260927 07:12:48.880116 21803 leader_election.cc:288] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:48.880165 21808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:48.880306 21808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [term 1 LEADER]: Becoming Leader. State: Replica: 0bb88e1a5c444928ace50b82b345e7e3, State: Running, Role: LEADER
I20260927 07:12:48.880393 21803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:48.880406 21808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb88e1a5c444928ace50b82b345e7e3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41825 } }
I20260927 07:12:48.880739 21808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bb88e1a5c444928ace50b82b345e7e3. Latest consensus state: current_term: 1 leader_uuid: "0bb88e1a5c444928ace50b82b345e7e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb88e1a5c444928ace50b82b345e7e3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41825 } } }
I20260927 07:12:48.880995 21808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:48.880927 21809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bb88e1a5c444928ace50b82b345e7e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb88e1a5c444928ace50b82b345e7e3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 41825 } } }
I20260927 07:12:48.881413 21809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:48.881539 21817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:48.882035 21817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:48.882501 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:48.872942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58710 (local address 127.4.174.126:41825)
0927 07:12:48.873114 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:48.873124 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:48.873155 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:12:48.873805 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:48.873811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:48.873858 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:48.873999 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:48.874597 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:48.875462 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:48.876260 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:48.876699 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:48.876797 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:48.879127 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:48.879132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:48.879132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:48.879144 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:48.880872 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:48.881253 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:48.881255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:48.881256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:48.881358 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:48.881547 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:48.881547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:48.881548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:48.881912 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:48.882054 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:48.882173 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 07:12:48.882301 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260927 07:12:48.883616 21817 catalog_manager.cc:1347] Generated new cluster ID: 33ef1b32b1a3437696e92765612d4579
I20260927 07:12:48.883656 21817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:48.904832 21817 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:48.905586 21817 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:48.918797 21817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3: Generated new TSK 0
I20260927 07:12:48.919392 21817 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:12:48.953050 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41825
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:49.031572 21829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:49.031786 21829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:49.031808 21829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:49.032963 21829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:49.033004 21829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:49.033030 21829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:12:49.034487 21829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:49.034741 21829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:49.034957 21829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:49.039798 21835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.039943 21829 server_base.cc:1034] running on GCE node
W20260927 07:12:49.039810 21834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.039817 21837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.040243 21829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:49.040500 21829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:49.041636 21829 hybrid_clock.cc:648] HybridClock initialized: now 1790493169041623 us; error 26 us; skew 500 ppm
Sep 27 07:12:49 dist-test-slave-zw7p krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493169, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:49.044199 21829 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:12:49.044637 21829 webserver.cc:460] Webserver started at http://127.4.174.65:35459/ using document root <none> and password file <none>
I20260927 07:12:49.044835 21829 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:49.044878 21829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:49.045001 21829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:49.046030 21829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "de757a90b6bb487fabfe5fa4ed48551c"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "7e69f031af6f728d2686ad2a8ec7b018"
server_key_iv: "69b6e6d45540dc212d39c14360e9a79a"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.046370 21829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "de757a90b6bb487fabfe5fa4ed48551c"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "7e69f031af6f728d2686ad2a8ec7b018"
server_key_iv: "69b6e6d45540dc212d39c14360e9a79a"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.047693 21829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 07:12:49.048421 21844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.048590 21829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:49.048668 21829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "de757a90b6bb487fabfe5fa4ed48551c"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "7e69f031af6f728d2686ad2a8ec7b018"
server_key_iv: "69b6e6d45540dc212d39c14360e9a79a"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.048733 21829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:49.060189 21829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:49.062198 21829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:49.062318 21829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:49.062516 21829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:49.062841 21829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:49.062875 21829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.062897 21829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:49.062929 21829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.068231 21829 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:46745
I20260927 07:12:49.068297 21957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:46745 every 8 connection(s)
I20260927 07:12:49.068558 21829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
Sep 27 07:12:49 dist-test-slave-zw7p krb5kdc[21722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493169, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:49.076705 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.069165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:53365 (local address 127.4.174.126:41825)
0927 07:12:49.069208 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.069210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.069415 (+ 205us) server_negotiation.cc:407] Connection header received
0927 07:12:49.069873 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.069874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.069882 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.069945 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:49.070863 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.071367 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.072314 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.072532 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.072582 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.074439 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:49.074440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:49.074441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:49.074447 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:49.075703 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:49.076028 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:49.076028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:49.076028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:49.076070 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:49.076247 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:49.076247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:49.076248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:49.076339 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:49.076380 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.076576 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.076640 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:12:49.076851 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.069138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41825 (local address 127.4.174.65:53365)
0927 07:12:49.069294 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.069334 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.069800 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.069998 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.070002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.070537 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:49.070773 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.070778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.071472 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.071473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.072240 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.072242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.072589 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.072590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.072678 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.073055 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:49.073063 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:49.074354 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:49.075766 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:49.075771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:49.075772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:49.075974 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:49.076124 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:49.076124 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:49.076124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:49.076207 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:49.076378 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:49.076381 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:49.076484 (+ 103us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.076558 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.076667 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260927 07:12:49.077282 21958 heartbeater.cc:347] Connected to a master server at 127.4.174.126:41825
I20260927 07:12:49.077359 21958 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:49.077541 21958 heartbeater.cc:511] Master 127.4.174.126:41825 requested a full tablet report, sending...
I20260927 07:12:49.077597 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 21829
I20260927 07:12:49.077692 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:12:49.077868 4793 external_mini_cluster.cc:1428] Setting key 5443da1b854558a70cac8700a4ed9a32
I20260927 07:12:49.078066 21767 ts_manager.cc:194] Registered new tserver with Master: de757a90b6bb487fabfe5fa4ed48551c (127.4.174.65:46745)
I20260927 07:12:49.078943 21767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:53365
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:12:49.100785 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41825
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:49.180518 21963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:49.180711 21963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:49.180732 21963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:49.181874 21963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:49.181914 21963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:49.181939 21963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:12:49.183346 21963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:49.183542 21963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:49.183725 21963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:49.188442 21968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.188473 21971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.188539 21963 server_base.cc:1034] running on GCE node
W20260927 07:12:49.188473 21969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.188800 21963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:49.189018 21963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:49.190171 21963 hybrid_clock.cc:648] HybridClock initialized: now 1790493169190158 us; error 26 us; skew 500 ppm
Sep 27 07:12:49 dist-test-slave-zw7p krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493169, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:49.192700 21963 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:12:49.193104 21963 webserver.cc:460] Webserver started at http://127.4.174.66:42553/ using document root <none> and password file <none>
I20260927 07:12:49.193300 21963 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:49.193344 21963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:49.193490 21963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:49.194507 21963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "6b5b08028afb489e8b4229e675e3d202"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "57d052801c630b1704098c94f05de18a"
server_key_iv: "262b23cff7ae14b6a8c3a131d4ce7c2c"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.194880 21963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "6b5b08028afb489e8b4229e675e3d202"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "57d052801c630b1704098c94f05de18a"
server_key_iv: "262b23cff7ae14b6a8c3a131d4ce7c2c"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.196166 21963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:49.196959 21978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.197124 21963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:49.197211 21963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "6b5b08028afb489e8b4229e675e3d202"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "57d052801c630b1704098c94f05de18a"
server_key_iv: "262b23cff7ae14b6a8c3a131d4ce7c2c"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.197278 21963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:49.213594 21963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:49.215682 21963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:49.215816 21963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:49.216025 21963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:49.216337 21963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:49.216370 21963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.216399 21963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:49.216423 21963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.221788 21963 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:39759
I20260927 07:12:49.221827 22091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:39759 every 8 connection(s)
I20260927 07:12:49.222133 21963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:12:49.225219 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 21963
I20260927 07:12:49.225318 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:12:49.225524 4793 external_mini_cluster.cc:1428] Setting key 7dfa78aa3649213d2e23a6beda77cba0
Sep 27 07:12:49 dist-test-slave-zw7p krb5kdc[21722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493169, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:49.231310 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.222809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:36457 (local address 127.4.174.126:41825)
0927 07:12:49.222851 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.222853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.223052 (+ 199us) server_negotiation.cc:407] Connection header received
0927 07:12:49.223556 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.223557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.223570 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.223616 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:49.224531 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.225049 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.226181 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.226371 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.226437 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.228562 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:49.228563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:49.228564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:49.228570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:49.229785 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:49.230299 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:49.230300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:49.230301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:49.230370 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:49.230672 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:49.230673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:49.230673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:49.230788 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:49.230844 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.231170 (+ 326us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.231244 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:12:49.231611 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.222781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41825 (local address 127.4.174.66:36457)
0927 07:12:49.222926 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.222969 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.223457 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.223685 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.223689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.224249 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:49.224438 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.224442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.225128 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.225129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.226103 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.226106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.226458 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.226459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.226636 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.227095 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:49.227104 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:49.228475 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:49.229861 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:49.229868 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:49.229870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:49.230186 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:49.230444 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:49.230445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:49.230446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:49.230610 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:49.230852 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:49.230857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:49.231033 (+ 176us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.231148 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.231316 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260927 07:12:49.232097 22092 heartbeater.cc:347] Connected to a master server at 127.4.174.126:41825
I20260927 07:12:49.232195 22092 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:49.232362 22092 heartbeater.cc:511] Master 127.4.174.126:41825 requested a full tablet report, sending...
I20260927 07:12:49.232760 21767 ts_manager.cc:194] Registered new tserver with Master: 6b5b08028afb489e8b4229e675e3d202 (127.4.174.66:39759)
I20260927 07:12:49.233116 21767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:36457
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:12:49.249639 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41825
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:49.330369 22097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:49.330624 22097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:49.330657 22097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:49.331823 22097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:49.331875 22097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:49.331908 22097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:12:49.333359 22097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:41825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:49.333580 22097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:49.333807 22097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:49.338627 22103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.338675 22102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.338650 22105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.338788 22097 server_base.cc:1034] running on GCE node
I20260927 07:12:49.338989 22097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:49.339195 22097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:49.340337 22097 hybrid_clock.cc:648] HybridClock initialized: now 1790493169340325 us; error 33 us; skew 500 ppm
Sep 27 07:12:49 dist-test-slave-zw7p krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493169, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:49.343068 22097 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:12:49.343513 22097 webserver.cc:460] Webserver started at http://127.4.174.67:36547/ using document root <none> and password file <none>
I20260927 07:12:49.343755 22097 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:49.343804 22097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:49.343923 22097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:49.344986 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "daf86b9b5b7d4387af7e17145f6a9f89"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "d68faad3eb55045ee04f5982fba289d6"
server_key_iv: "6bae9faebcddd6f407dbd7000356f221"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.345304 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "daf86b9b5b7d4387af7e17145f6a9f89"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "d68faad3eb55045ee04f5982fba289d6"
server_key_iv: "6bae9faebcddd6f407dbd7000356f221"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.346576 22097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:12:49.347218 22112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.347359 22097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:49.347445 22097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "daf86b9b5b7d4387af7e17145f6a9f89"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "d68faad3eb55045ee04f5982fba289d6"
server_key_iv: "6bae9faebcddd6f407dbd7000356f221"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.347500 22097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:49.371239 22097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:49.373211 22097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:49.373348 22097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:49.373567 22097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:49.373850 22097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:49.373881 22097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.373909 22097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:49.373932 22097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.379343 22097 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:40413
I20260927 07:12:49.379417 22225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:40413 every 8 connection(s)
I20260927 07:12:49.379714 22097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:12:49.384335 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 22097
I20260927 07:12:49.384423 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:12:49.384568 4793 external_mini_cluster.cc:1428] Setting key fca580f9c17f2e74ca6573a8d188a3fc
Sep 27 07:12:49 dist-test-slave-zw7p krb5kdc[21722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493169, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:49.388692 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.380412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:47267 (local address 127.4.174.126:41825)
0927 07:12:49.380482 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.380484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.380690 (+ 206us) server_negotiation.cc:407] Connection header received
0927 07:12:49.381207 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.381208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.381218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.381287 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:49.382241 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.382818 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.383845 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.384047 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.384118 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.386365 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:49.386366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:49.386366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:49.386374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:49.387598 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:49.387933 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:49.387934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:49.387934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:49.387982 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:49.388179 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:49.388180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:49.388181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:49.388301 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:49.388363 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.388566 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.388630 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 07:12:49.388845 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.380399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:41825 (local address 127.4.174.67:47267)
0927 07:12:49.380569 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.380612 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.381102 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.381360 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.381364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.381941 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:49.382138 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.382143 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.382902 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.382903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.383753 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.383756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.384118 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.384119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.384234 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.384769 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:49.384779 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:49.386250 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:49.387663 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:49.387669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:49.387670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:49.387878 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:49.388024 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:49.388024 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:49.388024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:49.388111 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:49.388356 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:49.388359 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:49.388471 (+ 112us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.388546 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.388660 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260927 07:12:49.389197 22226 heartbeater.cc:347] Connected to a master server at 127.4.174.126:41825
I20260927 07:12:49.389290 22226 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:49.389478 22226 heartbeater.cc:511] Master 127.4.174.126:41825 requested a full tablet report, sending...
I20260927 07:12:49.389787 21767 ts_manager.cc:194] Registered new tserver with Master: daf86b9b5b7d4387af7e17145f6a9f89 (127.4.174.67:40413)
I20260927 07:12:49.390120 21767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:47267
I20260927 07:12:49.396075 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:12:49.403100 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.397344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.4.174.126:41825)
0927 07:12:49.397430 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.397432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.397537 (+ 105us) server_negotiation.cc:407] Connection header received
0927 07:12:49.397602 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.397603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.397613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.397659 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:49.398200 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.398724 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.399434 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.399624 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.399678 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.400784 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:49.400786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:49.400786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:49.400793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:49.401997 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:49.402304 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:49.402304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:49.402305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:49.402358 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:49.402537 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:49.402538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:49.402538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:49.402728 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:49.402819 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.402936 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.403011 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 07:12:49.404668 21767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:12:49.406266 21767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:12:49.412424 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.408444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:40413 (local address 127.0.0.1:46846)
0927 07:12:49.408974 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.408983 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.409024 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.409268 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.409268 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.409270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:49.409303 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.409304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.410139 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.410140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.411352 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.411355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.412136 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.412137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.412214 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.412228 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.412291 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.412346 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":2,"thread_start_us":42,"threads_started":1}
I20260927 07:12:49.412431 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.408637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46846 (local address 127.4.174.67:40413)
0927 07:12:49.408905 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.408908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.409074 (+ 166us) server_negotiation.cc:407] Connection header received
0927 07:12:49.409116 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.409119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.409133 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.409196 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:49.409369 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.410052 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.411435 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.412060 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.412124 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.412184 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.412315 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.412355 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260927 07:12:49.412750 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.408415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39759 (local address 127.0.0.1:59920)
0927 07:12:49.408805 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.408814 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.408870 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.409121 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.409121 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.409123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:49.409187 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.409188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.410473 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.410474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.411529 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.411531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.412502 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.412503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.412583 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.412588 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.412640 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.412677 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260927 07:12:49.412782 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.408650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.4.174.66:39759)
0927 07:12:49.408816 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.408820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.408883 (+ 63us) server_negotiation.cc:407] Connection header received
0927 07:12:49.408933 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.408935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.408951 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.409033 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:49.409248 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.410360 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.411599 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.412334 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.412406 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.412467 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.412659 (+ 192us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.412706 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260927 07:12:49.413183 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.408497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45502 (local address 127.4.174.65:46745)
0927 07:12:49.408641 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.408646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.408858 (+ 212us) server_negotiation.cc:407] Connection header received
0927 07:12:49.408915 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.408917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.408934 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.409001 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:49.409235 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.410371 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.411684 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.412364 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.412431 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.412782 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.413061 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.413111 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260927 07:12:49.413395 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.408407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:46745 (local address 127.0.0.1:45502)
0927 07:12:49.408670 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.408708 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.408777 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.409093 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.409096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.409099 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:49.409161 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.409163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.410688 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.410689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.411601 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.411604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.412623 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.412624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.412676 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.412679 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.412714 (+ 35us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.412744 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":57,"threads_started":1}
I20260927 07:12:49.413331 22160 tablet_service.cc:1467] Processing CreateTablet for tablet b1dddf4ad8a94c23976b9e8a03a96db7 (DEFAULT_TABLE table=test-table [id=311c57f195244f119a65fdb3441f7560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:49.413451 22026 tablet_service.cc:1467] Processing CreateTablet for tablet b1dddf4ad8a94c23976b9e8a03a96db7 (DEFAULT_TABLE table=test-table [id=311c57f195244f119a65fdb3441f7560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:49.413700 22160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1dddf4ad8a94c23976b9e8a03a96db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:49.413739 22026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1dddf4ad8a94c23976b9e8a03a96db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:49.414052 21892 tablet_service.cc:1467] Processing CreateTablet for tablet b1dddf4ad8a94c23976b9e8a03a96db7 (DEFAULT_TABLE table=test-table [id=311c57f195244f119a65fdb3441f7560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:49.414307 21892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1dddf4ad8a94c23976b9e8a03a96db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:49.416996 22240 tablet_bootstrap.cc:492] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c: Bootstrap starting.
I20260927 07:12:49.417050 22242 tablet_bootstrap.cc:492] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Bootstrap starting.
I20260927 07:12:49.417207 22241 tablet_bootstrap.cc:492] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: Bootstrap starting.
I20260927 07:12:49.417675 22240 tablet_bootstrap.cc:654] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:49.417721 22242 tablet_bootstrap.cc:654] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:49.417757 22241 tablet_bootstrap.cc:654] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:49.418118 22240 log.cc:824] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:49.418259 22242 log.cc:824] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:49.418259 22241 log.cc:824] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:49.418972 22240 tablet_bootstrap.cc:492] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c: No bootstrap required, opened a new log
I20260927 07:12:49.418987 22242 tablet_bootstrap.cc:492] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: No bootstrap required, opened a new log
I20260927 07:12:49.419046 22242 ts_tablet_manager.cc:1397] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:49.419046 22240 ts_tablet_manager.cc:1397] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:49.419047 22241 tablet_bootstrap.cc:492] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: No bootstrap required, opened a new log
I20260927 07:12:49.419111 22241 ts_tablet_manager.cc:1397] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:49.420596 22241 raft_consensus.cc:348] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.420742 22241 raft_consensus.cc:374] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:49.420773 22241 raft_consensus.cc:729] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daf86b9b5b7d4387af7e17145f6a9f89, State: Initialized, Role: FOLLOWER
I20260927 07:12:49.420908 22241 consensus_queue.cc:260] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.420974 22240 raft_consensus.cc:348] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.420981 22242 raft_consensus.cc:348] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.421154 22240 raft_consensus.cc:374] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:49.421154 22242 raft_consensus.cc:374] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:49.421173 22241 ts_tablet_manager.cc:1428] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:12:49.421196 22240 raft_consensus.cc:729] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de757a90b6bb487fabfe5fa4ed48551c, State: Initialized, Role: FOLLOWER
I20260927 07:12:49.421196 22242 raft_consensus.cc:729] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b5b08028afb489e8b4229e675e3d202, State: Initialized, Role: FOLLOWER
I20260927 07:12:49.421286 22226 heartbeater.cc:503] Master 127.4.174.126:41825 was elected leader, sending a full tablet report...
I20260927 07:12:49.421306 22242 consensus_queue.cc:260] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.421306 22240 consensus_queue.cc:260] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.421581 22240 ts_tablet_manager.cc:1428] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 07:12:49.421581 22242 ts_tablet_manager.cc:1428] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 07:12:49.421650 22092 heartbeater.cc:503] Master 127.4.174.126:41825 was elected leader, sending a full tablet report...
I20260927 07:12:49.421706 21958 heartbeater.cc:503] Master 127.4.174.126:41825 was elected leader, sending a full tablet report...
I20260927 07:12:49.453876 22247 raft_consensus.cc:482] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:49.453997 22247 raft_consensus.cc:504] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.454335 22247 leader_election.cc:288] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de757a90b6bb487fabfe5fa4ed48551c (127.4.174.65:46745), daf86b9b5b7d4387af7e17145f6a9f89 (127.4.174.67:40413)
I20260927 07:12:49.457304 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.454493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:46745 (local address 127.4.174.66:41859)
0927 07:12:49.454547 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.454575 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.454621 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.454759 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.454760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.454762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:49.454837 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.454839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.455594 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.455595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.456446 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.456448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.457030 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.457031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.457113 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.457118 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.457182 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.457227 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 07:12:49.457346 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.454574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:41859 (local address 127.4.174.65:46745)
0927 07:12:49.454633 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.454635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.454644 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:49.454673 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.454674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.454684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.454741 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:49.454902 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.455519 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.456533 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.456957 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.457007 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.457014 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.457236 (+ 222us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.457280 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 07:12:49.457520 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.454706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:57033 (local address 127.4.174.67:40413)
0927 07:12:49.454755 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.454757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.454876 (+ 119us) server_negotiation.cc:407] Connection header received
0927 07:12:49.454939 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.454940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.454947 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.454983 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:49.455186 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.455748 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.456698 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.457141 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.457201 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.457210 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.457414 (+ 204us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.457461 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 07:12:49.457695 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.454624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:40413 (local address 127.4.174.66:57033)
0927 07:12:49.454820 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.454828 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.454872 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.455004 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.455005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.455007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:49.455070 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.455071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.455832 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.455833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.456612 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.456614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.457237 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.457238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.457313 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.457317 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.457589 (+ 272us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.457632 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":114,"threads_started":1}
I20260927 07:12:49.457695 21912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1dddf4ad8a94c23976b9e8a03a96db7" candidate_uuid: "6b5b08028afb489e8b4229e675e3d202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de757a90b6bb487fabfe5fa4ed48551c" is_pre_election: true
I20260927 07:12:49.457847 21912 raft_consensus.cc:2454] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b5b08028afb489e8b4229e675e3d202 in term 0.
I20260927 07:12:49.458029 22180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1dddf4ad8a94c23976b9e8a03a96db7" candidate_uuid: "6b5b08028afb489e8b4229e675e3d202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" is_pre_election: true
I20260927 07:12:49.458114 21982 leader_election.cc:302] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b5b08028afb489e8b4229e675e3d202, de757a90b6bb487fabfe5fa4ed48551c; no voters:
I20260927 07:12:49.458171 22180 raft_consensus.cc:2454] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b5b08028afb489e8b4229e675e3d202 in term 0.
I20260927 07:12:49.458282 22247 raft_consensus.cc:2781] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:12:49.458353 22247 raft_consensus.cc:482] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:12:49.458377 22247 raft_consensus.cc:3037] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:49.459117 22247 raft_consensus.cc:504] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.459303 22247 leader_election.cc:288] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [CANDIDATE]: Term 1 election: Requested vote from peers de757a90b6bb487fabfe5fa4ed48551c (127.4.174.65:46745), daf86b9b5b7d4387af7e17145f6a9f89 (127.4.174.67:40413)
I20260927 07:12:49.459388 22180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1dddf4ad8a94c23976b9e8a03a96db7" candidate_uuid: "6b5b08028afb489e8b4229e675e3d202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf86b9b5b7d4387af7e17145f6a9f89"
I20260927 07:12:49.459473 22180 raft_consensus.cc:3037] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:49.460156 22180 raft_consensus.cc:2454] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b5b08028afb489e8b4229e675e3d202 in term 1.
I20260927 07:12:49.460256 21912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1dddf4ad8a94c23976b9e8a03a96db7" candidate_uuid: "6b5b08028afb489e8b4229e675e3d202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de757a90b6bb487fabfe5fa4ed48551c"
I20260927 07:12:49.460306 21981 leader_election.cc:302] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b5b08028afb489e8b4229e675e3d202, daf86b9b5b7d4387af7e17145f6a9f89; no voters:
I20260927 07:12:49.460337 21912 raft_consensus.cc:3037] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:49.460377 22247 raft_consensus.cc:2781] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:49.460522 22247 raft_consensus.cc:686] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 1 LEADER]: Becoming Leader. State: Replica: 6b5b08028afb489e8b4229e675e3d202, State: Running, Role: LEADER
I20260927 07:12:49.460620 22247 consensus_queue.cc:237] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } }
I20260927 07:12:49.461187 21912 raft_consensus.cc:2454] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b5b08028afb489e8b4229e675e3d202 in term 1.
I20260927 07:12:49.461470 21765 catalog_manager.cc:5495] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b5b08028afb489e8b4229e675e3d202 (127.4.174.66). New cstate: current_term: 1 leader_uuid: "6b5b08028afb489e8b4229e675e3d202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b5b08028afb489e8b4229e675e3d202" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 } health_report { overall_health: UNKNOWN } } }
I20260927 07:12:49.469146 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.467198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.4.174.66:39759)
0927 07:12:49.467246 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.467247 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.467255 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:12:49.467283 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.467283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.467298 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.467356 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:12:49.467452 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.467888 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.468576 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.468754 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.468803 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.468907 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:12:49.469001 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:12:49.469055 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.469066 (+ 11us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.469082 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 07:12:49.470711 21912 raft_consensus.cc:1261] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 1 FOLLOWER]: Refusing update from remote peer 6b5b08028afb489e8b4229e675e3d202: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:49.470718 22180 raft_consensus.cc:1261] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 1 FOLLOWER]: Refusing update from remote peer 6b5b08028afb489e8b4229e675e3d202: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:49.470988 22247 consensus_queue.cc:1035] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [LEADER]: Connected to new peer: Peer: permanent_uuid: "daf86b9b5b7d4387af7e17145f6a9f89" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 40413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:49.471124 22250 consensus_queue.cc:1035] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de757a90b6bb487fabfe5fa4ed48551c" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 46745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 07:12:49.472642 22093 tablet.cc:2369] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:12:49.475463 21767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:49.475543 21767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:49.476226 21767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bb88e1a5c444928ace50b82b345e7e3: Sending DeleteTablet for 3 replicas of tablet b1dddf4ad8a94c23976b9e8a03a96db7
I20260927 07:12:49.476548 21892 tablet_service.cc:1514] Processing DeleteTablet for tablet b1dddf4ad8a94c23976b9e8a03a96db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:49 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:45502
I20260927 07:12:49.476574 22160 tablet_service.cc:1514] Processing DeleteTablet for tablet b1dddf4ad8a94c23976b9e8a03a96db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:49 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:46846
I20260927 07:12:49.476565 22026 tablet_service.cc:1514] Processing DeleteTablet for tablet b1dddf4ad8a94c23976b9e8a03a96db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:49 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:59920
I20260927 07:12:49.476771 22266 tablet_replica.cc:330] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c: stopping tablet replica
I20260927 07:12:49.476861 22264 tablet_replica.cc:330] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: stopping tablet replica
I20260927 07:12:49.476958 22266 raft_consensus.cc:2229] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:49.476965 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 21829
I20260927 07:12:49.477044 22264 raft_consensus.cc:2229] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:12:49.477058 22266 raft_consensus.cc:2258] T b1dddf4ad8a94c23976b9e8a03a96db7 P de757a90b6bb487fabfe5fa4ed48551c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:49.477190 22264 raft_consensus.cc:2258] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:49.477762 22265 tablet_replica.cc:330] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: stopping tablet replica
I20260927 07:12:49.477856 22265 raft_consensus.cc:2229] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:49.477957 22265 raft_consensus.cc:2258] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:49.478329 22265 ts_tablet_manager.cc:1905] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:49.478997 22264 ts_tablet_manager.cc:1905] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:49.479887 22265 ts_tablet_manager.cc:1918] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:49.479954 22265 log.cc:1196] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/b1dddf4ad8a94c23976b9e8a03a96db7
I20260927 07:12:49.480199 22265 ts_tablet_manager.cc:1939] T b1dddf4ad8a94c23976b9e8a03a96db7 P daf86b9b5b7d4387af7e17145f6a9f89: Deleting consensus metadata
I20260927 07:12:49.480540 21753 catalog_manager.cc:4841] TS daf86b9b5b7d4387af7e17145f6a9f89 (127.4.174.67:40413): tablet b1dddf4ad8a94c23976b9e8a03a96db7 (table test-table [id=311c57f195244f119a65fdb3441f7560]) successfully deleted
I20260927 07:12:49.480939 22264 ts_tablet_manager.cc:1918] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:49.481097 22264 log.cc:1196] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/b1dddf4ad8a94c23976b9e8a03a96db7
I20260927 07:12:49.481411 22264 ts_tablet_manager.cc:1939] T b1dddf4ad8a94c23976b9e8a03a96db7 P 6b5b08028afb489e8b4229e675e3d202: Deleting consensus metadata
I20260927 07:12:49.481829 21752 catalog_manager.cc:4841] TS 6b5b08028afb489e8b4229e675e3d202 (127.4.174.66:39759): tablet b1dddf4ad8a94c23976b9e8a03a96db7 (table test-table [id=311c57f195244f119a65fdb3441f7560]) successfully deleted
W20260927 07:12:49.483139 21754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:46745 (error 108)
I20260927 07:12:49.483616 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.483467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:46745 (local address 127.0.0.1:45512)
0927 07:12:49.483553 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.483588 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:46745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260927 07:12:49.483727 21754 catalog_manager.cc:4584] TS de757a90b6bb487fabfe5fa4ed48551c (127.4.174.65:46745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1dddf4ad8a94c23976b9e8a03a96db7: Network error: Client connection negotiation failed: client connection to 127.4.174.65:46745: connect: Connection refused (error 111)
I20260927 07:12:49.483767 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 21963
I20260927 07:12:49.487756 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 22097
I20260927 07:12:49.491854 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 21735
2026-09-27T07:12:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1790 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-27T07:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:49Z Disabled control of system clock
I20260927 07:12:49.517000 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35597
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:35597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:12:49.596802 22271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:49.597025 22271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:49.597044 22271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:49.598141 22271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:49.598174 22271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:49.598186 22271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:49.598196 22271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:49.598206 22271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:49.599512 22271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:35597
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:49.599704 22271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:49.599934 22271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:49.604530 22276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.604593 22271 server_base.cc:1034] running on GCE node
W20260927 07:12:49.604539 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.604539 22279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.604946 22271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:49.605248 22271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:49.606431 22271 hybrid_clock.cc:648] HybridClock initialized: now 1790493169606415 us; error 29 us; skew 500 ppm
I20260927 07:12:49.610471 22271 webserver.cc:460] Webserver started at http://127.4.174.126:45127/ using document root <none> and password file <none>
I20260927 07:12:49.610723 22271 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:49.610774 22271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:49.610900 22271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:49.611953 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "aa047c93f03b409f9697262ae3a77645"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "897babd6fd780efff277943ad018a6f0"
server_key_iv: "6d57781e0b8ea205bc68cb8cddbb7627"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.612284 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "aa047c93f03b409f9697262ae3a77645"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "897babd6fd780efff277943ad018a6f0"
server_key_iv: "6d57781e0b8ea205bc68cb8cddbb7627"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.613515 22271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 07:12:49.614187 22285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.614331 22271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.614428 22271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "aa047c93f03b409f9697262ae3a77645"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "897babd6fd780efff277943ad018a6f0"
server_key_iv: "6d57781e0b8ea205bc68cb8cddbb7627"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.614495 22271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:49.624590 22271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:49.624837 22271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:49.624950 22271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:49.628489 22271 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:35597
I20260927 07:12:49.628527 22337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:35597 every 8 connection(s)
I20260927 07:12:49.628813 22271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:49.629702 22338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:49.631379 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 22271
I20260927 07:12:49.631469 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:49.631635 4793 external_mini_cluster.cc:1428] Setting key a35181fcd75224d5d85dbe10fa328cda
I20260927 07:12:49.631983 22338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645: Bootstrap starting.
I20260927 07:12:49.632565 22338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:49.632825 22338 log.cc:824] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:49.633433 22338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645: No bootstrap required, opened a new log
I20260927 07:12:49.635476 22338 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa047c93f03b409f9697262ae3a77645" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35597 } }
I20260927 07:12:49.635603 22338 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:49.635634 22338 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa047c93f03b409f9697262ae3a77645, State: Initialized, Role: FOLLOWER
I20260927 07:12:49.635778 22338 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa047c93f03b409f9697262ae3a77645" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35597 } }
I20260927 07:12:49.635867 22338 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:49.635917 22338 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:49.635964 22338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:49.636520 22338 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa047c93f03b409f9697262ae3a77645" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35597 } }
I20260927 07:12:49.636606 22338 leader_election.cc:302] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa047c93f03b409f9697262ae3a77645; no voters:
I20260927 07:12:49.636759 22338 leader_election.cc:288] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:49.636829 22343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:49.636986 22343 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [term 1 LEADER]: Becoming Leader. State: Replica: aa047c93f03b409f9697262ae3a77645, State: Running, Role: LEADER
I20260927 07:12:49.637036 22338 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:49.637066 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.632107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.4.174.126:35597)
0927 07:12:49.632276 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.632283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.632307 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:12:49.632725 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.632733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.632793 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.632907 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:49.633159 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.634094 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.635220 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.635628 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.635748 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.636360 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:49.636364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:49.636364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:49.636521 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:49.636553 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:49.636563 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:49.636625 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.636737 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.636819 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260927 07:12:49.637089 22343 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa047c93f03b409f9697262ae3a77645" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35597 } }
I20260927 07:12:49.637707 22345 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa047c93f03b409f9697262ae3a77645. Latest consensus state: current_term: 1 leader_uuid: "aa047c93f03b409f9697262ae3a77645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa047c93f03b409f9697262ae3a77645" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35597 } } }
I20260927 07:12:49.637727 22344 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa047c93f03b409f9697262ae3a77645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa047c93f03b409f9697262ae3a77645" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35597 } } }
I20260927 07:12:49.637894 22344 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:49.637861 22345 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:49.638293 22355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:49.638784 22355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:49.640270 22355 catalog_manager.cc:1347] Generated new cluster ID: f03d7ce9e6c44861961d4b786606d8d3
I20260927 07:12:49.640316 22355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:49.644145 22355 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:49.644752 22355 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:49.650921 22355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645: Generated new TSK 0
I20260927 07:12:49.651082 22355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:12:49.653025 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35597
--builtin_ntp_servers=127.4.174.84:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:12:49.732887 22362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:49.733093 22362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:49.733114 22362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:49.734239 22362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:49.734283 22362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:49.734308 22362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:12:49.735704 22362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:49.735934 22362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:49.736239 22362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:49.741206 22367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.741242 22368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.741382 22370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.741633 22362 server_base.cc:1034] running on GCE node
I20260927 07:12:49.741832 22362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:49.742089 22362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:49.743238 22362 hybrid_clock.cc:648] HybridClock initialized: now 1790493169743225 us; error 28 us; skew 500 ppm
I20260927 07:12:49.744522 22362 webserver.cc:460] Webserver started at http://127.4.174.65:39327/ using document root <none> and password file <none>
I20260927 07:12:49.744760 22362 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:49.744813 22362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:49.744944 22362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:49.746238 22362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "a2cd63cf25454a07be9a24a0798d3be3"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "faa7e4388a4c9232224e73a2704c8de2"
server_key_iv: "222d67586dbbf05d2e84962939da405c"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.746632 22362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "a2cd63cf25454a07be9a24a0798d3be3"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "faa7e4388a4c9232224e73a2704c8de2"
server_key_iv: "222d67586dbbf05d2e84962939da405c"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.748095 22362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:12:49.748885 22376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.749037 22362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:12:49.749114 22362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "a2cd63cf25454a07be9a24a0798d3be3"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "faa7e4388a4c9232224e73a2704c8de2"
server_key_iv: "222d67586dbbf05d2e84962939da405c"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.749166 22362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:49.756405 22362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:49.756654 22362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:49.756769 22362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:49.756955 22362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:49.757274 22362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:49.757316 22362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.757350 22362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:49.757373 22362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.762544 22362 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:45099
I20260927 07:12:49.762629 22489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:45099 every 8 connection(s)
I20260927 07:12:49.762946 22362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:12:49.767535 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 22362
I20260927 07:12:49.767640 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:12:49.767750 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.763588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:39101 (local address 127.4.174.126:35597)
0927 07:12:49.763633 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.763635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.763866 (+ 231us) server_negotiation.cc:407] Connection header received
0927 07:12:49.764355 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.764357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.764374 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.764424 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:49.765035 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.765544 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.766466 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.766705 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.766753 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.767490 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:49.767491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:49.767492 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:49.767497 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:49.767515 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:49.767520 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:49.767579 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.767648 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.767674 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:12:49.767854 4793 external_mini_cluster.cc:1428] Setting key d08dce12a066b818086459885a66a7c8
I20260927 07:12:49.767980 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.763579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35597 (local address 127.4.174.65:39101)
0927 07:12:49.763742 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.763774 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.764250 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.764486 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.764489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.764496 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:49.764973 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.764977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.765608 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.765609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.766388 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.766390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.766767 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.766768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.766878 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.767359 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:49.767359 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:49.767387 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:49.767411 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:49.767581 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:49.767583 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:49.767585 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.767641 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.767746 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260927 07:12:49.768471 22490 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35597
I20260927 07:12:49.768553 22490 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:49.768724 22490 heartbeater.cc:511] Master 127.4.174.126:35597 requested a full tablet report, sending...
I20260927 07:12:49.768738 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35597
--builtin_ntp_servers=127.4.174.84:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:12:49.769249 22302 ts_manager.cc:194] Registered new tserver with Master: a2cd63cf25454a07be9a24a0798d3be3 (127.4.174.65:45099)
I20260927 07:12:49.770117 22302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.65:39101
W20260927 07:12:49.847342 22493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:49.847548 22493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:49.847570 22493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:49.848656 22493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:49.848688 22493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:49.848712 22493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:12:49.849983 22493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:49.850178 22493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:49.850420 22493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:49.855218 22498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.855271 22501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.855271 22499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.855453 22493 server_base.cc:1034] running on GCE node
I20260927 07:12:49.855636 22493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:49.855845 22493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:49.857007 22493 hybrid_clock.cc:648] HybridClock initialized: now 1790493169856992 us; error 29 us; skew 500 ppm
I20260927 07:12:49.858251 22493 webserver.cc:460] Webserver started at http://127.4.174.66:35229/ using document root <none> and password file <none>
I20260927 07:12:49.858484 22493 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:49.858532 22493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:49.858680 22493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:49.859706 22493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "0a30678152ff4751a4803dd9258ba03f"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "83f2204b0f911f2e34724adbb5b5ba95"
server_key_iv: "dcfb73911dab1c521074b8b9d25a4f8a"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.860046 22493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "0a30678152ff4751a4803dd9258ba03f"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "83f2204b0f911f2e34724adbb5b5ba95"
server_key_iv: "dcfb73911dab1c521074b8b9d25a4f8a"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.861225 22493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:12:49.861902 22507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.862053 22493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:49.862129 22493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "0a30678152ff4751a4803dd9258ba03f"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "83f2204b0f911f2e34724adbb5b5ba95"
server_key_iv: "dcfb73911dab1c521074b8b9d25a4f8a"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.862193 22493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:49.873693 22493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:49.873960 22493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:49.874078 22493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:49.874279 22493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:49.874619 22493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:49.874665 22493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.874696 22493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:49.874718 22493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.879865 22493 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:34319
I20260927 07:12:49.879913 22620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:34319 every 8 connection(s)
I20260927 07:12:49.880194 22493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:12:49.883550 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 22493
I20260927 07:12:49.883638 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:12:49.883843 4793 external_mini_cluster.cc:1428] Setting key a9d80a6125bb35041e5860f19f9f90bf
I20260927 07:12:49.884785 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35597
--builtin_ntp_servers=127.4.174.84:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:12:49.885142 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.880822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:34983 (local address 127.4.174.126:35597)
0927 07:12:49.880877 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:49.880879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:49.881085 (+ 206us) server_negotiation.cc:407] Connection header received
0927 07:12:49.881567 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:49.881568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:49.881576 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:49.881634 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:49.882234 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.882773 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.883707 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.883993 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.884053 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.884892 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:49.884894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:49.884894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:49.884899 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:49.884917 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:49.884921 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:49.884970 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:49.885044 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 07:12:49.885066 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:12:49.885344 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:49.880798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35597 (local address 127.4.174.66:34983)
0927 07:12:49.880969 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:49.881007 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:49.881491 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:49.881696 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:49.881699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:49.881708 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:49.882177 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.882180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.882832 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.882833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.883627 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:49.883629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:49.884072 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:49.884073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:49.884224 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:49.884752 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:49.884752 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:49.884779 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:49.884800 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:49.884977 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:49.884984 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:49.884986 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:12:49.885046 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:12:49.885142 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260927 07:12:49.885805 22621 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35597
I20260927 07:12:49.885885 22621 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:49.886080 22621 heartbeater.cc:511] Master 127.4.174.126:35597 requested a full tablet report, sending...
I20260927 07:12:49.886447 22302 ts_manager.cc:194] Registered new tserver with Master: 0a30678152ff4751a4803dd9258ba03f (127.4.174.66:34319)
I20260927 07:12:49.886816 22302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.66:34983
W20260927 07:12:49.962667 22624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:49.962852 22624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:49.962879 22624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:49.963999 22624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:49.964035 22624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:49.964059 22624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:12:49.965394 22624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:49.965610 22624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:49.965880 22624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:49.970583 22630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.970613 22632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:49.970584 22629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:49.970775 22624 server_base.cc:1034] running on GCE node
I20260927 07:12:49.971053 22624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:49.971257 22624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:49.972412 22624 hybrid_clock.cc:648] HybridClock initialized: now 1790493169972378 us; error 29 us; skew 500 ppm
I20260927 07:12:49.973582 22624 webserver.cc:460] Webserver started at http://127.4.174.67:39725/ using document root <none> and password file <none>
I20260927 07:12:49.973800 22624 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:49.973843 22624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:49.973963 22624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:49.975008 22624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "b338354e4eb3496c98a30740a9cd479b"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "4ad1d61e0939e84ec17e6c9ebeb14942"
server_key_iv: "79b63d4edd954399768b80ae80a977a1"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.975315 22624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "b338354e4eb3496c98a30740a9cd479b"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "4ad1d61e0939e84ec17e6c9ebeb14942"
server_key_iv: "79b63d4edd954399768b80ae80a977a1"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.976526 22624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 07:12:49.977209 22638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:49.977357 22624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:49.977438 22624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "b338354e4eb3496c98a30740a9cd479b"
format_stamp: "Formatted at 2026-09-27 07:12:49 on dist-test-slave-zw7p"
server_key: "4ad1d61e0939e84ec17e6c9ebeb14942"
server_key_iv: "79b63d4edd954399768b80ae80a977a1"
server_key_version: "encryptionkey@0"
I20260927 07:12:49.977506 22624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:50.002367 22624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:50.002722 22624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:50.002867 22624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:50.003083 22624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:50.003419 22624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:50.003453 22624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:50.003475 22624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:50.003487 22624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:50.008693 22624 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:38757
I20260927 07:12:50.008755 22751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:38757 every 8 connection(s)
I20260927 07:12:50.009059 22624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:12:50.009718 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 22624
I20260927 07:12:50.009802 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:12:50.010028 4793 external_mini_cluster.cc:1428] Setting key 60fbfc342313c264eb5446b4949b6368
I20260927 07:12:50.014326 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.009872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:58129 (local address 127.4.174.126:35597)
0927 07:12:50.009927 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:50.009930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:50.010232 (+ 302us) server_negotiation.cc:407] Connection header received
0927 07:12:50.010903 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:50.010904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:50.010915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:50.010966 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:50.011667 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.012147 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.013123 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.013301 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.013362 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.014150 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:50.014151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:50.014151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:50.014154 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:50.014164 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:50.014167 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:50.014198 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:50.014252 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 07:12:50.014269 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:12:50.014596 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.009864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35597 (local address 127.4.174.67:58129)
0927 07:12:50.010086 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:50.010132 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:50.010798 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:50.011029 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:50.011032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:50.011041 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:50.011532 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.011536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.012247 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.012249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.013058 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.013060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.013393 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.013394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.013531 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.014035 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:50.014036 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:50.014065 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:50.014088 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:50.014200 (+ 112us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:50.014202 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:50.014205 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:50.014248 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 07:12:50.014325 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260927 07:12:50.014984 22752 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35597
I20260927 07:12:50.015069 22752 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:50.015239 22752 heartbeater.cc:511] Master 127.4.174.126:35597 requested a full tablet report, sending...
I20260927 07:12:50.015555 22302 ts_manager.cc:194] Registered new tserver with Master: b338354e4eb3496c98a30740a9cd479b (127.4.174.67:38757)
I20260927 07:12:50.015895 22302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.67:58129
I20260927 07:12:50.021733 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:12:50.025929 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.023162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39140 (local address 127.4.174.126:35597)
0927 07:12:50.023211 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:50.023214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:50.023238 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:12:50.023307 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:50.023308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:50.023317 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:50.023366 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:50.023545 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.024028 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.024755 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.024948 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.025000 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.025685 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:50.025686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:50.025686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:50.025691 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:50.025705 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:50.025708 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:50.025749 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:50.025824 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 07:12:50.025850 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:12:50.027356 22302 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:12:50.028890 22302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:12:50.035068 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.031075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:34319 (local address 127.0.0.1:50534)
0927 07:12:50.031414 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:50.031454 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:50.031533 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:50.031728 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:50.031731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:50.031733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:50.031802 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.031804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.032774 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.032775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.033804 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.033807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.034747 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.034748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.034824 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.034849 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 07:12:50.034926 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 07:12:50.034981 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":71,"threads_started":1}
I20260927 07:12:50.035075 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.031166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.4.174.66:34319)
0927 07:12:50.031430 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:50.031434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:50.031515 (+ 81us) server_negotiation.cc:407] Connection header received
0927 07:12:50.031617 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:50.031619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:50.031636 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:50.031716 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:50.031901 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.032697 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.033912 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.034677 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.034744 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.034839 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:50.034941 (+ 102us) server_negotiation.cc:299] Negotiation successful
0927 07:12:50.034980 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":96,"threads_started":1}
I20260927 07:12:50.035233 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.031075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45099 (local address 127.0.0.1:36424)
0927 07:12:50.031414 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:50.031454 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:50.031533 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:50.031928 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:50.031929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:50.031931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:50.031976 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.031978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.033279 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.033280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.034181 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.034183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.035031 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.035032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.035092 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.035097 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:12:50.035124 (+ 27us) client_negotiation.cc:240] Negotiation successful
0927 07:12:50.035161 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":35,"spinlock_wait_cycles":768,"thread_start_us":88,"threads_started":1}
I20260927 07:12:50.035555 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.031277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:38757 (local address 127.0.0.1:59528)
0927 07:12:50.031646 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:50.031654 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:50.031767 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:50.032038 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:50.032039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:50.032041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:50.032079 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.032081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.033282 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.033283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.034380 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.034383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.035355 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.035356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.035412 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.035416 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:12:50.035459 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 07:12:50.035494 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":24,"thread_start_us":55,"threads_started":1}
I20260927 07:12:50.035646 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.031173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.4.174.65:45099)
0927 07:12:50.031344 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:50.031348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:50.031538 (+ 190us) server_negotiation.cc:407] Connection header received
0927 07:12:50.031810 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:50.031813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:50.031828 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:50.032042 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:50.032091 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.033181 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.034256 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.034936 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.035406 (+ 470us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.035500 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:50.035533 (+ 33us) server_negotiation.cc:299] Negotiation successful
0927 07:12:50.035571 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260927 07:12:50.035802 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.031483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.4.174.67:38757)
0927 07:12:50.031633 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:50.031636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:50.031704 (+ 68us) server_negotiation.cc:407] Connection header received
0927 07:12:50.031881 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:50.031884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:50.031900 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:50.031972 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:50.032158 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.033204 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.034466 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.035178 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.035233 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.035322 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:50.035696 (+ 374us) server_negotiation.cc:299] Negotiation successful
0927 07:12:50.035735 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260927 07:12:50.036167 22555 tablet_service.cc:1467] Processing CreateTablet for tablet 89bd266d8c3145adb310b69c2e6abe1d (DEFAULT_TABLE table=test-table [id=990680a4e13e4fabab205dd95f96fe3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:50.036285 22424 tablet_service.cc:1467] Processing CreateTablet for tablet 89bd266d8c3145adb310b69c2e6abe1d (DEFAULT_TABLE table=test-table [id=990680a4e13e4fabab205dd95f96fe3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:50.036574 22555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bd266d8c3145adb310b69c2e6abe1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:50.036581 22424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bd266d8c3145adb310b69c2e6abe1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:50.040117 22767 tablet_bootstrap.cc:492] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Bootstrap starting.
I20260927 07:12:50.040508 22686 tablet_service.cc:1467] Processing CreateTablet for tablet 89bd266d8c3145adb310b69c2e6abe1d (DEFAULT_TABLE table=test-table [id=990680a4e13e4fabab205dd95f96fe3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:50.040796 22767 tablet_bootstrap.cc:654] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:50.040843 22686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bd266d8c3145adb310b69c2e6abe1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:50.041080 22767 log.cc:824] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:50.041375 22766 tablet_bootstrap.cc:492] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3: Bootstrap starting.
I20260927 07:12:50.041829 22767 tablet_bootstrap.cc:492] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: No bootstrap required, opened a new log
I20260927 07:12:50.041882 22766 tablet_bootstrap.cc:654] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:50.041913 22767 ts_tablet_manager.cc:1397] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:50.042145 22766 log.cc:824] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:50.043012 22766 tablet_bootstrap.cc:492] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3: No bootstrap required, opened a new log
I20260927 07:12:50.043207 22766 ts_tablet_manager.cc:1397] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:50.044472 22770 tablet_bootstrap.cc:492] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: Bootstrap starting.
I20260927 07:12:50.044571 22767 raft_consensus.cc:348] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.044749 22767 raft_consensus.cc:374] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:50.044790 22767 raft_consensus.cc:729] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a30678152ff4751a4803dd9258ba03f, State: Initialized, Role: FOLLOWER
I20260927 07:12:50.044914 22767 consensus_queue.cc:260] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.045192 22767 ts_tablet_manager.cc:1428] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260927 07:12:50.045228 22621 heartbeater.cc:503] Master 127.4.174.126:35597 was elected leader, sending a full tablet report...
I20260927 07:12:50.045434 22770 tablet_bootstrap.cc:654] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:50.045446 22766 raft_consensus.cc:348] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.045607 22766 raft_consensus.cc:374] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:50.045646 22766 raft_consensus.cc:729] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cd63cf25454a07be9a24a0798d3be3, State: Initialized, Role: FOLLOWER
I20260927 07:12:50.045761 22770 log.cc:824] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:50.045758 22766 consensus_queue.cc:260] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.046020 22766 ts_tablet_manager.cc:1428] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 07:12:50.046135 22490 heartbeater.cc:503] Master 127.4.174.126:35597 was elected leader, sending a full tablet report...
I20260927 07:12:50.046466 22770 tablet_bootstrap.cc:492] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: No bootstrap required, opened a new log
I20260927 07:12:50.046540 22770 ts_tablet_manager.cc:1397] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:50.048496 22770 raft_consensus.cc:348] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.048625 22770 raft_consensus.cc:374] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:50.048650 22770 raft_consensus.cc:729] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b338354e4eb3496c98a30740a9cd479b, State: Initialized, Role: FOLLOWER
I20260927 07:12:50.048748 22770 consensus_queue.cc:260] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.049001 22770 ts_tablet_manager.cc:1428] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:12:50.049046 22752 heartbeater.cc:503] Master 127.4.174.126:35597 was elected leader, sending a full tablet report...
I20260927 07:12:50.116322 22772 raft_consensus.cc:482] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:50.116448 22772 raft_consensus.cc:504] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.116760 22772 leader_election.cc:288] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b338354e4eb3496c98a30740a9cd479b (127.4.174.67:38757), 0a30678152ff4751a4803dd9258ba03f (127.4.174.66:34319)
I20260927 07:12:50.119906 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.116980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:38757 (local address 127.4.174.65:58451)
0927 07:12:50.117044 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:50.117055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:50.117103 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:50.117297 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:50.117298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:50.117300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:50.117405 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.117407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.118325 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.118326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.119134 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.119136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.119688 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.119689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.119757 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.119760 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:50.119810 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:12:50.119843 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":24}
I20260927 07:12:50.119978 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.117115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:58451 (local address 127.4.174.67:38757)
0927 07:12:50.117181 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:50.117183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:50.117192 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:50.117217 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:50.117218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:50.117226 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:50.117265 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:50.117476 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.118224 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.119219 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.119618 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.119673 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.119680 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:50.119851 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 07:12:50.119904 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 07:12:50.120077 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.117024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:51319 (local address 127.4.174.66:34319)
0927 07:12:50.117071 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:50.117074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:50.117305 (+ 231us) server_negotiation.cc:407] Connection header received
0927 07:12:50.117341 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:50.117341 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:50.117349 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:50.117388 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:50.117519 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.118241 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.119308 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.119681 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.119755 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.119763 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:50.119972 (+ 209us) server_negotiation.cc:299] Negotiation successful
0927 07:12:50.120018 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:12:50.120085 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.116980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:34319 (local address 127.4.174.65:51319)
0927 07:12:50.117212 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:50.117221 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:50.117298 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:50.117401 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:50.117401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:50.117403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:50.117461 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.117462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.118332 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.118333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.119220 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:50.119223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.119783 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.119784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:50.119875 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.119880 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:12:50.119946 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 07:12:50.119984 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":15,"thread_start_us":120,"threads_started":1}
I20260927 07:12:50.120338 22706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bd266d8c3145adb310b69c2e6abe1d" candidate_uuid: "a2cd63cf25454a07be9a24a0798d3be3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b338354e4eb3496c98a30740a9cd479b" is_pre_election: true
I20260927 07:12:50.120404 22575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bd266d8c3145adb310b69c2e6abe1d" candidate_uuid: "a2cd63cf25454a07be9a24a0798d3be3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a30678152ff4751a4803dd9258ba03f" is_pre_election: true
I20260927 07:12:50.120501 22706 raft_consensus.cc:2454] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2cd63cf25454a07be9a24a0798d3be3 in term 0.
I20260927 07:12:50.120548 22575 raft_consensus.cc:2454] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2cd63cf25454a07be9a24a0798d3be3 in term 0.
I20260927 07:12:50.120748 22378 leader_election.cc:302] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2cd63cf25454a07be9a24a0798d3be3, b338354e4eb3496c98a30740a9cd479b; no voters:
I20260927 07:12:50.120910 22772 raft_consensus.cc:2781] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:12:50.120972 22772 raft_consensus.cc:482] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:12:50.120988 22772 raft_consensus.cc:3037] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:50.121671 22772 raft_consensus.cc:504] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.121793 22772 leader_election.cc:288] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [CANDIDATE]: Term 1 election: Requested vote from peers b338354e4eb3496c98a30740a9cd479b (127.4.174.67:38757), 0a30678152ff4751a4803dd9258ba03f (127.4.174.66:34319)
I20260927 07:12:50.121984 22706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bd266d8c3145adb310b69c2e6abe1d" candidate_uuid: "a2cd63cf25454a07be9a24a0798d3be3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b338354e4eb3496c98a30740a9cd479b"
I20260927 07:12:50.121984 22575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bd266d8c3145adb310b69c2e6abe1d" candidate_uuid: "a2cd63cf25454a07be9a24a0798d3be3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a30678152ff4751a4803dd9258ba03f"
I20260927 07:12:50.122071 22706 raft_consensus.cc:3037] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:50.122071 22575 raft_consensus.cc:3037] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:50.123013 22706 raft_consensus.cc:2454] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2cd63cf25454a07be9a24a0798d3be3 in term 1.
I20260927 07:12:50.123013 22575 raft_consensus.cc:2454] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2cd63cf25454a07be9a24a0798d3be3 in term 1.
I20260927 07:12:50.123165 22378 leader_election.cc:302] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2cd63cf25454a07be9a24a0798d3be3, b338354e4eb3496c98a30740a9cd479b; no voters:
I20260927 07:12:50.123277 22772 raft_consensus.cc:2781] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:50.123438 22772 raft_consensus.cc:686] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [term 1 LEADER]: Becoming Leader. State: Replica: a2cd63cf25454a07be9a24a0798d3be3, State: Running, Role: LEADER
I20260927 07:12:50.123529 22772 consensus_queue.cc:237] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } }
I20260927 07:12:50.124320 22301 catalog_manager.cc:5495] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2cd63cf25454a07be9a24a0798d3be3 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "a2cd63cf25454a07be9a24a0798d3be3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cd63cf25454a07be9a24a0798d3be3" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 } health_report { overall_health: UNKNOWN } } }
W20260927 07:12:50.130736 22622 tablet.cc:2369] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:12:50.150149 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.147962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.4.174.65:45099)
0927 07:12:50.148022 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:50.148024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:50.148033 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:50.148074 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:50.148075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:50.148101 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:50.148161 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:12:50.148266 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.148723 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.149462 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:50.149648 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:50.149699 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:50.149835 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:12:50.149944 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:12:50.150015 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:50.150034 (+ 19us) server_negotiation.cc:299] Negotiation successful
0927 07:12:50.150052 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 07:12:50.152045 22706 raft_consensus.cc:1261] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 1 FOLLOWER]: Refusing update from remote peer a2cd63cf25454a07be9a24a0798d3be3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:50.152045 22575 raft_consensus.cc:1261] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 1 FOLLOWER]: Refusing update from remote peer a2cd63cf25454a07be9a24a0798d3be3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:50.152474 22772 consensus_queue.cc:1035] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b338354e4eb3496c98a30740a9cd479b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:50.152652 22776 consensus_queue.cc:1035] T 89bd266d8c3145adb310b69c2e6abe1d P a2cd63cf25454a07be9a24a0798d3be3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a30678152ff4751a4803dd9258ba03f" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 34319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:50.155021 22779 mvcc.cc:205] Tried to move back new op lower bound from 7333860024940011520 to 7333860024826621952. Current Snapshot: MvccSnapshot[applied={T|T < 7333860024940011520}]
I20260927 07:12:50.157307 22301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:50.157410 22301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:50.158174 22301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa047c93f03b409f9697262ae3a77645: Sending DeleteTablet for 3 replicas of tablet 89bd266d8c3145adb310b69c2e6abe1d
I20260927 07:12:50.158486 22424 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bd266d8c3145adb310b69c2e6abe1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:50 UTC) from {username='slave'} at 127.0.0.1:36424
I20260927 07:12:50.158535 22686 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bd266d8c3145adb310b69c2e6abe1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:50 UTC) from {username='slave'} at 127.0.0.1:59528
I20260927 07:12:50.158779 22790 tablet_replica.cc:330] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: stopping tablet replica
I20260927 07:12:50.158780 22555 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bd266d8c3145adb310b69c2e6abe1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:50 UTC) from {username='slave'} at 127.0.0.1:50534
I20260927 07:12:50.158874 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 22362
I20260927 07:12:50.158987 22790 raft_consensus.cc:2229] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:50.159121 22790 raft_consensus.cc:2258] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:50.159548 22791 tablet_replica.cc:330] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: stopping tablet replica
I20260927 07:12:50.159653 22791 raft_consensus.cc:2229] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:50.159759 22791 raft_consensus.cc:2258] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:50.160126 22791 ts_tablet_manager.cc:1905] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:50.160193 22790 ts_tablet_manager.cc:1905] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:50.161726 22791 ts_tablet_manager.cc:1918] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:50.161793 22791 log.cc:1196] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/89bd266d8c3145adb310b69c2e6abe1d
I20260927 07:12:50.161839 22790 ts_tablet_manager.cc:1918] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:50.161909 22790 log.cc:1196] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/89bd266d8c3145adb310b69c2e6abe1d
I20260927 07:12:50.162055 22791 ts_tablet_manager.cc:1939] T 89bd266d8c3145adb310b69c2e6abe1d P 0a30678152ff4751a4803dd9258ba03f: Deleting consensus metadata
I20260927 07:12:50.162147 22790 ts_tablet_manager.cc:1939] T 89bd266d8c3145adb310b69c2e6abe1d P b338354e4eb3496c98a30740a9cd479b: Deleting consensus metadata
I20260927 07:12:50.162441 22288 catalog_manager.cc:4841] TS 0a30678152ff4751a4803dd9258ba03f (127.4.174.66:34319): tablet 89bd266d8c3145adb310b69c2e6abe1d (table test-table [id=990680a4e13e4fabab205dd95f96fe3b]) successfully deleted
I20260927 07:12:50.162489 22287 catalog_manager.cc:4841] TS b338354e4eb3496c98a30740a9cd479b (127.4.174.67:38757): tablet 89bd266d8c3145adb310b69c2e6abe1d (table test-table [id=990680a4e13e4fabab205dd95f96fe3b]) successfully deleted
W20260927 07:12:50.167086 22287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:45099 (error 108)
I20260927 07:12:50.167338 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 22493
I20260927 07:12:50.167842 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:50.167415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45099 (local address 127.0.0.1:36442)
0927 07:12:50.167767 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:50.167810 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:45099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332}
W20260927 07:12:50.167959 22287 catalog_manager.cc:4584] TS a2cd63cf25454a07be9a24a0798d3be3 (127.4.174.65:45099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89bd266d8c3145adb310b69c2e6abe1d: Network error: Client connection negotiation failed: client connection to 127.4.174.65:45099: connect: Connection refused (error 111)
I20260927 07:12:50.172039 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 22624
I20260927 07:12:50.175918 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 22271
2026-09-27T07:12:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (683 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:12:50 dist-test-slave-zw7p krb5kdc[22795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:12:50 dist-test-slave-zw7p krb5kdc[22795](info): set up 2 sockets
Sep 27 07:12:50 dist-test-slave-zw7p krb5kdc[22795](info): commencing operation
krb5kdc: starting...
W20260927 07:12:51.122089 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:12:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:12:51.233803 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:46083
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:46083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:51.312306 22808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:51.312513 22808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:51.312536 22808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:51.313638 22808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:51.313673 22808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:51.313685 22808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:51.313695 22808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:51.313706 22808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:51.315178 22808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:46083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:46083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:51.315379 22808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:51.315553 22808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:51.320127 22814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:51.320132 22813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:51.320284 22808 server_base.cc:1034] running on GCE node
W20260927 07:12:51.320144 22816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:51.320675 22808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:51.320969 22808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:51.322108 22808 hybrid_clock.cc:648] HybridClock initialized: now 1790493171322095 us; error 30 us; skew 500 ppm
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:51.324470 22808 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:12:51.324899 22808 webserver.cc:460] Webserver started at http://127.4.174.126:45433/ using document root <none> and password file <none>
I20260927 07:12:51.325103 22808 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:51.325146 22808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:51.325266 22808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:51.326310 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "354dbf65c8d74011813fd9407e010bac"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "f7a18529d8d0c4ed366f07d31c89fdb6"
server_key_iv: "beee45b83d96a85f6d87da8bcf19a879"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.326664 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "354dbf65c8d74011813fd9407e010bac"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "f7a18529d8d0c4ed366f07d31c89fdb6"
server_key_iv: "beee45b83d96a85f6d87da8bcf19a879"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.327881 22808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:51.328611 22823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.328769 22808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:51.328851 22808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "354dbf65c8d74011813fd9407e010bac"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "f7a18529d8d0c4ed366f07d31c89fdb6"
server_key_iv: "beee45b83d96a85f6d87da8bcf19a879"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.328907 22808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:51.337627 22808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:51.339522 22808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:51.339655 22808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:51.343330 22808 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:46083
I20260927 07:12:51.343380 22875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:46083 every 8 connection(s)
I20260927 07:12:51.343696 22808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:51.344476 22876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:51.346805 22876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac: Bootstrap starting.
I20260927 07:12:51.347330 22876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:51.347580 22876 log.cc:824] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:51.348088 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 22808
I20260927 07:12:51.348176 22876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac: No bootstrap required, opened a new log
I20260927 07:12:51.348186 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:51.348356 4793 external_mini_cluster.cc:1428] Setting key dd8baf03f2faeec71c452df936a3d79c
I20260927 07:12:51.350178 22876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354dbf65c8d74011813fd9407e010bac" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46083 } }
I20260927 07:12:51.350359 22876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:51.350401 22876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354dbf65c8d74011813fd9407e010bac, State: Initialized, Role: FOLLOWER
I20260927 07:12:51.350533 22876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354dbf65c8d74011813fd9407e010bac" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46083 } }
I20260927 07:12:51.350636 22876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:51.350673 22876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:51.350716 22876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:51.351363 22876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354dbf65c8d74011813fd9407e010bac" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46083 } }
I20260927 07:12:51.351466 22876 leader_election.cc:302] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 354dbf65c8d74011813fd9407e010bac; no voters:
I20260927 07:12:51.351604 22876 leader_election.cc:288] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:51.351637 22881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:51.351779 22881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [term 1 LEADER]: Becoming Leader. State: Replica: 354dbf65c8d74011813fd9407e010bac, State: Running, Role: LEADER
I20260927 07:12:51.351817 22876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:51.351871 22881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354dbf65c8d74011813fd9407e010bac" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46083 } }
I20260927 07:12:51.352190 22881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 354dbf65c8d74011813fd9407e010bac. Latest consensus state: current_term: 1 leader_uuid: "354dbf65c8d74011813fd9407e010bac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354dbf65c8d74011813fd9407e010bac" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46083 } } }
I20260927 07:12:51.352293 22881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:51.352192 22883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "354dbf65c8d74011813fd9407e010bac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354dbf65c8d74011813fd9407e010bac" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 46083 } } }
I20260927 07:12:51.352381 22883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:51.352491 22888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:51.352980 22888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:51.354411 22888 catalog_manager.cc:1347] Generated new cluster ID: 9b16f5ce6862458fbfb928136b360f9c
I20260927 07:12:51.354460 22888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:51.360237 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.348802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43502 (local address 127.4.174.126:46083)
0927 07:12:51.348975 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.348985 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.349016 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:12:51.349452 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.349462 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.349524 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.349661 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:51.350222 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.351824 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.352952 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.353596 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.353699 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.356436 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:51.356439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:51.356440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:51.356452 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:51.358360 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.358803 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.358807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.358807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.358937 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.359166 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.359166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.359167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.359516 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:51.359685 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.359758 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.360019 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260927 07:12:51.363080 22888 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:51.363654 22888 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:51.366310 22888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac: Generated new TSK 0
I20260927 07:12:51.366470 22888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:12:51.391175 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:46083
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:51.471139 22902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:51.471354 22902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:51.471377 22902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:51.472543 22902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:51.472589 22902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:51.472615 22902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:12:51.474063 22902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:51.474277 22902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:51.474498 22902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:51.479373 22907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:51.479521 22908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:51.479610 22902 server_base.cc:1034] running on GCE node
W20260927 07:12:51.479379 22910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:51.479868 22902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:51.480114 22902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:51.481281 22902 hybrid_clock.cc:648] HybridClock initialized: now 1790493171481267 us; error 29 us; skew 500 ppm
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:51.483771 22902 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:12:51.484174 22902 webserver.cc:460] Webserver started at http://127.4.174.65:44519/ using document root <none> and password file <none>
I20260927 07:12:51.484383 22902 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:51.484428 22902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:51.484516 22902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:51.485522 22902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "8d92ac458e3f45399e12fbf31fe554f5"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "4edd8f8caa81cf733a3d21b45ddd18e8"
server_key_iv: "b19e88bc09daf41116e273fe12651138"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.485806 22902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "8d92ac458e3f45399e12fbf31fe554f5"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "4edd8f8caa81cf733a3d21b45ddd18e8"
server_key_iv: "b19e88bc09daf41116e273fe12651138"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.487030 22902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:12:51.487722 22917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.487888 22902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:12:51.487962 22902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "8d92ac458e3f45399e12fbf31fe554f5"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "4edd8f8caa81cf733a3d21b45ddd18e8"
server_key_iv: "b19e88bc09daf41116e273fe12651138"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.488016 22902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:51.504842 22902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:51.507014 22902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:51.507135 22902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:51.507320 22902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:51.507596 22902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:51.507637 22902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.507661 22902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:51.507673 22902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.513187 22902 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:38545
I20260927 07:12:51.513248 23030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:38545 every 8 connection(s)
I20260927 07:12:51.513538 22902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:12:51.515503 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 22902
I20260927 07:12:51.515591 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:12:51.515762 4793 external_mini_cluster.cc:1428] Setting key 64f7a5a680abe55910170b9e77f732c2
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
I20260927 07:12:51.523782 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.514209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:49551 (local address 127.4.174.126:46083)
0927 07:12:51.514257 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.514259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.514471 (+ 212us) server_negotiation.cc:407] Connection header received
0927 07:12:51.515020 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.515022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.515034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.515085 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:51.516137 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.516746 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.518070 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.518416 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.518495 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.520817 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:51.520818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:51.520818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:51.520825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:51.522021 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.522529 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.522531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.522531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.522668 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.522991 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.522992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.522993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.523154 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:51.523221 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.523591 (+ 370us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.523682 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:12:51.524339 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.514186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:46083 (local address 127.4.174.65:49551)
0927 07:12:51.514326 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.514387 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:51.514919 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:51.515155 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:51.515160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:51.515757 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:51.516037 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.516044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.516846 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.516847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.517979 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.517982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.518493 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.518494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.518626 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.519098 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:51.519106 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:51.520729 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:51.522110 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:51.522118 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:51.522120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:51.522425 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:51.522762 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:51.522763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:51.522764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:51.522909 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:51.523231 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:51.523238 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:51.523413 (+ 175us) client_negotiation.cc:769] Sending connection context
0927 07:12:51.523916 (+ 503us) client_negotiation.cc:240] Negotiation successful
0927 07:12:51.524101 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260927 07:12:51.524923 23031 heartbeater.cc:347] Connected to a master server at 127.4.174.126:46083
I20260927 07:12:51.525014 23031 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:51.525204 23031 heartbeater.cc:511] Master 127.4.174.126:46083 requested a full tablet report, sending...
I20260927 07:12:51.525779 22840 ts_manager.cc:194] Registered new tserver with Master: 8d92ac458e3f45399e12fbf31fe554f5 (127.4.174.65:38545)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
I20260927 07:12:51.526599 22840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:49551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:12:51.539710 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:46083
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:51.619257 23036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:51.619445 23036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:51.619465 23036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:51.620625 23036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:51.620667 23036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:51.620693 23036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:12:51.622133 23036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:51.622347 23036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:51.622581 23036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:51.627261 23041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:51.627280 23044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:51.627270 23042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:51.627511 23036 server_base.cc:1034] running on GCE node
I20260927 07:12:51.627693 23036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:51.627897 23036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:51.629043 23036 hybrid_clock.cc:648] HybridClock initialized: now 1790493171629028 us; error 29 us; skew 500 ppm
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:51.631579 23036 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:12:51.632052 23036 webserver.cc:460] Webserver started at http://127.4.174.66:44275/ using document root <none> and password file <none>
I20260927 07:12:51.632272 23036 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:51.632315 23036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:51.632433 23036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:51.633460 23036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "cb8ab375bb40410fa9b7639429f1c989"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "396842459f524e1f7f9f39128349ce2d"
server_key_iv: "d36ec2d3c3aa3c952cb7efbbe76156c9"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.633772 23036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "cb8ab375bb40410fa9b7639429f1c989"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "396842459f524e1f7f9f39128349ce2d"
server_key_iv: "d36ec2d3c3aa3c952cb7efbbe76156c9"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.635090 23036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:12:51.635815 23051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.635963 23036 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:12:51.636041 23036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "cb8ab375bb40410fa9b7639429f1c989"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "396842459f524e1f7f9f39128349ce2d"
server_key_iv: "d36ec2d3c3aa3c952cb7efbbe76156c9"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.636119 23036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:51.665109 23036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:51.666920 23036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:51.667043 23036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:51.667275 23036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:51.667583 23036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:51.667615 23036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.667639 23036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:51.667662 23036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.672808 23036 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:33857
I20260927 07:12:51.672857 23164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:33857 every 8 connection(s)
I20260927 07:12:51.673166 23036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:12:51.673940 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 23036
I20260927 07:12:51.674063 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:12:51.674276 4793 external_mini_cluster.cc:1428] Setting key 1342686fb578643555b51338a963e407
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
I20260927 07:12:51.682783 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.673843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:42637 (local address 127.4.174.126:46083)
0927 07:12:51.673897 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.673899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.674149 (+ 250us) server_negotiation.cc:407] Connection header received
0927 07:12:51.674793 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.674794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.674804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.674856 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:51.676084 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.676673 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.677923 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.678112 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.678167 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.680388 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:51.680389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:51.680390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:51.680405 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:51.681586 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.681935 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.681936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.681936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.681990 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.682192 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.682193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.682194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.682298 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:51.682352 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.682614 (+ 262us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.682695 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:12:51.682968 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.673826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:46083 (local address 127.4.174.66:42637)
0927 07:12:51.674014 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.674067 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:51.674697 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:51.674941 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:51.674945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:51.675697 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:51.675974 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.675981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.676765 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.676766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.677848 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.677851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.678187 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.678188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.678318 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.678828 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:51.678837 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:51.680302 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:51.681655 (+ 1353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:51.681661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:51.681663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:51.681867 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:51.682042 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:51.682042 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:51.682043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:51.682141 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:51.682349 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:51.682355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:51.682486 (+ 131us) client_negotiation.cc:769] Sending connection context
0927 07:12:51.682591 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 07:12:51.682744 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260927 07:12:51.683398 23165 heartbeater.cc:347] Connected to a master server at 127.4.174.126:46083
I20260927 07:12:51.683495 23165 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:51.683668 23165 heartbeater.cc:511] Master 127.4.174.126:46083 requested a full tablet report, sending...
I20260927 07:12:51.684000 22840 ts_manager.cc:194] Registered new tserver with Master: cb8ab375bb40410fa9b7639429f1c989 (127.4.174.66:33857)
I20260927 07:12:51.684410 22840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:42637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:12:51.698462 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:46083
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:51.776998 23170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:51.777208 23170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:51.777230 23170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:51.778328 23170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:51.778364 23170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:51.778388 23170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:12:51.779788 23170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:51.779973 23170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:51.780151 23170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:51.784819 23176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:51.784832 23178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:51.785012 23175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:51.784922 23170 server_base.cc:1034] running on GCE node
I20260927 07:12:51.785187 23170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:51.785394 23170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:51.786548 23170 hybrid_clock.cc:648] HybridClock initialized: now 1790493171786534 us; error 28 us; skew 500 ppm
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:51.788852 23170 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:12:51.789247 23170 webserver.cc:460] Webserver started at http://127.4.174.67:35727/ using document root <none> and password file <none>
I20260927 07:12:51.789424 23170 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:51.789463 23170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:51.789551 23170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:51.790519 23170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "d2f1754c88894881a53c6eabd613b62b"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "1940cec2528df1e0b470c3f147845482"
server_key_iv: "0d8fca891aa92b3976af8c9c6c2ec196"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.790887 23170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "d2f1754c88894881a53c6eabd613b62b"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "1940cec2528df1e0b470c3f147845482"
server_key_iv: "0d8fca891aa92b3976af8c9c6c2ec196"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.792088 23170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:51.792814 23185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.792990 23170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:51.793080 23170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "d2f1754c88894881a53c6eabd613b62b"
format_stamp: "Formatted at 2026-09-27 07:12:51 on dist-test-slave-zw7p"
server_key: "1940cec2528df1e0b470c3f147845482"
server_key_iv: "0d8fca891aa92b3976af8c9c6c2ec196"
server_key_version: "encryptionkey@0"
I20260927 07:12:51.793136 23170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:51.801200 23170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:51.802841 23170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:51.802942 23170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:51.803102 23170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:51.803331 23170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:51.803369 23170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.803395 23170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:51.803411 23170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:51.808545 23170 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:39461
I20260927 07:12:51.808611 23298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:39461 every 8 connection(s)
I20260927 07:12:51.808866 23170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:12:51.812883 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 23170
I20260927 07:12:51.812974 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:12:51.813153 4793 external_mini_cluster.cc:1428] Setting key 336ae4e878a7dbca9e5ae9db6dae7ea8
Sep 27 07:12:51 dist-test-slave-zw7p krb5kdc[22795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493171, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:51.816877 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.809518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:53719 (local address 127.4.174.126:46083)
0927 07:12:51.809577 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.809578 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.809737 (+ 159us) server_negotiation.cc:407] Connection header received
0927 07:12:51.810188 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.810189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.810198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.810256 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:51.811154 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.811643 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.812544 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.812737 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.812795 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.814825 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:51.814826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:51.814826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:51.814833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:51.815854 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.816188 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.816188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.816189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.816233 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.816427 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.816427 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.816428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.816521 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:51.816566 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.816760 (+ 194us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.816816 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 07:12:51.817021 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.809486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:46083 (local address 127.4.174.67:53719)
0927 07:12:51.809624 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.809663 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:51.810113 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:51.810307 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:51.810310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:51.810898 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:51.811084 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.811089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.811706 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.811707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.812480 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.812483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.812805 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.812806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.812895 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.813336 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:51.813369 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:51.814749 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:51.815915 (+ 1166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:51.815920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:51.815922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:51.816134 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:51.816275 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:51.816276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:51.816276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:51.816385 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:51.816562 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:51.816565 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:51.816669 (+ 104us) client_negotiation.cc:769] Sending connection context
0927 07:12:51.816741 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 07:12:51.816853 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260927 07:12:51.817468 23299 heartbeater.cc:347] Connected to a master server at 127.4.174.126:46083
I20260927 07:12:51.817564 23299 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:51.817703 23299 heartbeater.cc:511] Master 127.4.174.126:46083 requested a full tablet report, sending...
I20260927 07:12:51.818011 22840 ts_manager.cc:194] Registered new tserver with Master: d2f1754c88894881a53c6eabd613b62b (127.4.174.67:39461)
I20260927 07:12:51.818378 22840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:53719
I20260927 07:12:51.824795 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:12:51.831869 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.826095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.4.174.126:46083)
0927 07:12:51.826142 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.826143 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.826247 (+ 104us) server_negotiation.cc:407] Connection header received
0927 07:12:51.826289 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.826290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.826298 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.826341 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:51.826875 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.827342 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.828028 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.828197 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.828239 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.829519 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:51.829520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:51.829520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:51.829526 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:51.830754 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.831146 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.831146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.831147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.831204 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:51.831441 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:51.831442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:51.831442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:51.831593 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:51.831666 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.831751 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.831796 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 07:12:51.833374 22840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:12:51.834949 22840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:12:51.841157 23307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.837146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:38545 (local address 127.0.0.1:59976)
0927 07:12:51.837380 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.837387 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:51.837421 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:51.837585 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:51.837588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:51.837590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:51.837669 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.837671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.838790 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.838792 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.840115 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.840118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.840862 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.840863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.840939 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.840953 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 07:12:51.841017 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 07:12:51.841075 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":61,"threads_started":1}
I20260927 07:12:51.841168 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.837262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59976 (local address 127.4.174.65:38545)
0927 07:12:51.837436 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.837441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.837450 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:51.837481 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.837483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.837496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.837557 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:51.837734 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.838710 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.840202 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.840787 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.840855 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.840919 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.841047 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.841091 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260927 07:12:51.841326 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.836943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39461 (local address 127.0.0.1:46318)
0927 07:12:51.837063 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.837100 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:51.837161 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:51.837930 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:51.837930 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:51.837933 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:51.837971 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.837973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.839324 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.839325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.840207 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.840209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.841119 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.841120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.841182 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.841185 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:51.841213 (+ 28us) client_negotiation.cc:240] Negotiation successful
0927 07:12:51.841251 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
I20260927 07:12:51.841475 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.837270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33766 (local address 127.4.174.66:33857)
0927 07:12:51.837697 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.837701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.837710 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:51.837747 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.837749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.837763 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.837833 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:51.837968 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.839092 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.840313 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.841024 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.841089 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.841145 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.841360 (+ 215us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.841404 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":71,"threads_started":1}
I20260927 07:12:51.841502 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.836949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:33857 (local address 127.0.0.1:33766)
0927 07:12:51.837257 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.837267 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:51.837314 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:51.837873 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:51.837874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:51.837876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:51.837913 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.837914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.839192 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.839193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.840237 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.840238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.841172 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.841173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.841224 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.841227 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:51.841265 (+ 38us) client_negotiation.cc:240] Negotiation successful
0927 07:12:51.841296 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":54,"threads_started":1}
I20260927 07:12:51.841867 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.837070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.4.174.67:39461)
0927 07:12:51.837457 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.837462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.837472 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:12:51.837508 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.837510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.837522 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.838052 (+ 530us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:51.838118 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.839239 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.840284 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.841024 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.841673 (+ 649us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.841735 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.841757 (+ 22us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.841796 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":82,"threads_started":1}
I20260927 07:12:51.842207 23099 tablet_service.cc:1467] Processing CreateTablet for tablet 660ffe5bc04440a8b02ab6154caf6d50 (DEFAULT_TABLE table=test-table [id=dd90f063904946fbb3bb82b44e0bc149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:51.842308 22965 tablet_service.cc:1467] Processing CreateTablet for tablet 660ffe5bc04440a8b02ab6154caf6d50 (DEFAULT_TABLE table=test-table [id=dd90f063904946fbb3bb82b44e0bc149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:51.842458 23233 tablet_service.cc:1467] Processing CreateTablet for tablet 660ffe5bc04440a8b02ab6154caf6d50 (DEFAULT_TABLE table=test-table [id=dd90f063904946fbb3bb82b44e0bc149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:51.842550 23099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 660ffe5bc04440a8b02ab6154caf6d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:51.842639 22965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 660ffe5bc04440a8b02ab6154caf6d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:51.842747 23233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 660ffe5bc04440a8b02ab6154caf6d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:51.845290 23313 tablet_bootstrap.cc:492] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5: Bootstrap starting.
I20260927 07:12:51.845643 23314 tablet_bootstrap.cc:492] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: Bootstrap starting.
I20260927 07:12:51.846123 23313 tablet_bootstrap.cc:654] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:51.846230 23314 tablet_bootstrap.cc:654] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:51.846757 23313 log.cc:824] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:51.846802 23314 log.cc:824] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:51.847100 23315 tablet_bootstrap.cc:492] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: Bootstrap starting.
I20260927 07:12:51.847425 23314 tablet_bootstrap.cc:492] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: No bootstrap required, opened a new log
I20260927 07:12:51.847484 23314 ts_tablet_manager.cc:1397] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 07:12:51.847615 23315 tablet_bootstrap.cc:654] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:51.848040 23315 log.cc:824] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:51.848743 23315 tablet_bootstrap.cc:492] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: No bootstrap required, opened a new log
I20260927 07:12:51.848795 23315 ts_tablet_manager.cc:1397] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:51.849025 23314 raft_consensus.cc:348] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.849160 23314 raft_consensus.cc:374] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:51.849190 23314 raft_consensus.cc:729] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f1754c88894881a53c6eabd613b62b, State: Initialized, Role: FOLLOWER
I20260927 07:12:51.849301 23314 consensus_queue.cc:260] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.849558 23314 ts_tablet_manager.cc:1428] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:12:51.849576 23299 heartbeater.cc:503] Master 127.4.174.126:46083 was elected leader, sending a full tablet report...
I20260927 07:12:51.850132 23313 tablet_bootstrap.cc:492] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5: No bootstrap required, opened a new log
I20260927 07:12:51.850205 23313 ts_tablet_manager.cc:1397] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260927 07:12:51.850533 23315 raft_consensus.cc:348] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.850741 23315 raft_consensus.cc:374] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:51.850778 23315 raft_consensus.cc:729] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb8ab375bb40410fa9b7639429f1c989, State: Initialized, Role: FOLLOWER
I20260927 07:12:51.850890 23315 consensus_queue.cc:260] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.851154 23315 ts_tablet_manager.cc:1428] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 07:12:51.851919 23313 raft_consensus.cc:348] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.852062 23313 raft_consensus.cc:374] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:51.852084 23313 raft_consensus.cc:729] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d92ac458e3f45399e12fbf31fe554f5, State: Initialized, Role: FOLLOWER
I20260927 07:12:51.852195 23313 consensus_queue.cc:260] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.852438 23313 ts_tablet_manager.cc:1428] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260927 07:12:51.852483 23031 heartbeater.cc:503] Master 127.4.174.126:46083 was elected leader, sending a full tablet report...
I20260927 07:12:51.852560 23165 heartbeater.cc:503] Master 127.4.174.126:46083 was elected leader, sending a full tablet report...
I20260927 07:12:51.881524 23321 raft_consensus.cc:482] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:51.881644 23321 raft_consensus.cc:504] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.882033 23321 leader_election.cc:288] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2f1754c88894881a53c6eabd613b62b (127.4.174.67:39461), cb8ab375bb40410fa9b7639429f1c989 (127.4.174.66:33857)
I20260927 07:12:51.884785 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.882312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:37233 (local address 127.4.174.66:33857)
0927 07:12:51.882353 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.882355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.882467 (+ 112us) server_negotiation.cc:407] Connection header received
0927 07:12:51.882595 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.882596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.882607 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.882653 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:51.882794 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.883235 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.884120 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.884478 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.884534 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.884553 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.884685 (+ 132us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.884724 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:12:51.884806 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.882266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:42319 (local address 127.4.174.67:39461)
0927 07:12:51.882305 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.882307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.882315 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:12:51.882369 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.882370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.882386 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.882433 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:51.882644 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.883177 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.884117 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.884486 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.884546 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.884554 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.884695 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.884739 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:12:51.884815 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.882233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:33857 (local address 127.4.174.65:37233)
0927 07:12:51.882418 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.882425 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:51.882490 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:51.882673 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:51.882674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:51.882675 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:51.882741 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.882743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.883304 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.883305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.884039 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.884041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.884545 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.884546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.884615 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.884620 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:12:51.884667 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 07:12:51.884711 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260927 07:12:51.884814 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.882197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:39461 (local address 127.4.174.65:42319)
0927 07:12:51.882251 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.882263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:51.882307 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:51.882488 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:51.882489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:51.882491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:51.882578 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.882580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.883250 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.883251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.884021 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:51.884024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.884560 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.884561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:51.884633 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.884636 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:51.884686 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:12:51.884732 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":45}
I20260927 07:12:51.885135 23119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "660ffe5bc04440a8b02ab6154caf6d50" candidate_uuid: "8d92ac458e3f45399e12fbf31fe554f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb8ab375bb40410fa9b7639429f1c989" is_pre_election: true
I20260927 07:12:51.885208 23253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "660ffe5bc04440a8b02ab6154caf6d50" candidate_uuid: "8d92ac458e3f45399e12fbf31fe554f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f1754c88894881a53c6eabd613b62b" is_pre_election: true
I20260927 07:12:51.885280 23119 raft_consensus.cc:2454] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d92ac458e3f45399e12fbf31fe554f5 in term 0.
I20260927 07:12:51.885376 23253 raft_consensus.cc:2454] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d92ac458e3f45399e12fbf31fe554f5 in term 0.
I20260927 07:12:51.885480 22920 leader_election.cc:302] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d92ac458e3f45399e12fbf31fe554f5, cb8ab375bb40410fa9b7639429f1c989; no voters:
I20260927 07:12:51.885632 23321 raft_consensus.cc:2781] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:12:51.885689 23321 raft_consensus.cc:482] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:12:51.885715 23321 raft_consensus.cc:3037] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:51.886404 23321 raft_consensus.cc:504] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.886519 23321 leader_election.cc:288] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [CANDIDATE]: Term 1 election: Requested vote from peers d2f1754c88894881a53c6eabd613b62b (127.4.174.67:39461), cb8ab375bb40410fa9b7639429f1c989 (127.4.174.66:33857)
I20260927 07:12:51.886660 23253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "660ffe5bc04440a8b02ab6154caf6d50" candidate_uuid: "8d92ac458e3f45399e12fbf31fe554f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f1754c88894881a53c6eabd613b62b"
I20260927 07:12:51.886667 23119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "660ffe5bc04440a8b02ab6154caf6d50" candidate_uuid: "8d92ac458e3f45399e12fbf31fe554f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb8ab375bb40410fa9b7639429f1c989"
I20260927 07:12:51.886732 23119 raft_consensus.cc:3037] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:51.886734 23253 raft_consensus.cc:3037] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:51.887392 23119 raft_consensus.cc:2454] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d92ac458e3f45399e12fbf31fe554f5 in term 1.
I20260927 07:12:51.887400 23253 raft_consensus.cc:2454] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d92ac458e3f45399e12fbf31fe554f5 in term 1.
I20260927 07:12:51.887557 22921 leader_election.cc:302] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d92ac458e3f45399e12fbf31fe554f5, d2f1754c88894881a53c6eabd613b62b; no voters:
I20260927 07:12:51.887661 23321 raft_consensus.cc:2781] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:51.887784 23321 raft_consensus.cc:686] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [term 1 LEADER]: Becoming Leader. State: Replica: 8d92ac458e3f45399e12fbf31fe554f5, State: Running, Role: LEADER
I20260927 07:12:51.887856 23321 consensus_queue.cc:237] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } }
I20260927 07:12:51.888597 22838 catalog_manager.cc:5495] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d92ac458e3f45399e12fbf31fe554f5 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "8d92ac458e3f45399e12fbf31fe554f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d92ac458e3f45399e12fbf31fe554f5" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 38545 } health_report { overall_health: HEALTHY } } }
I20260927 07:12:51.898137 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.896184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.4.174.65:38545)
0927 07:12:51.896239 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:51.896240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:51.896248 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:12:51.896279 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:51.896280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:51.896295 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:51.896374 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:12:51.896450 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.896906 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.897590 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:51.897762 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:51.897803 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:51.897899 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:12:51.897980 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:12:51.898041 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:51.898061 (+ 20us) server_negotiation.cc:299] Negotiation successful
0927 07:12:51.898069 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 07:12:51.899806 23253 raft_consensus.cc:1261] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 1 FOLLOWER]: Refusing update from remote peer 8d92ac458e3f45399e12fbf31fe554f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:51.899814 23119 raft_consensus.cc:1261] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 1 FOLLOWER]: Refusing update from remote peer 8d92ac458e3f45399e12fbf31fe554f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:51.900116 23321 consensus_queue.cc:1035] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2f1754c88894881a53c6eabd613b62b" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 39461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:51.900246 23323 consensus_queue.cc:1035] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb8ab375bb40410fa9b7639429f1c989" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:51.902486 23326 mvcc.cc:205] Tried to move back new op lower bound from 7333860032099168256 to 7333860032053260288. Current Snapshot: MvccSnapshot[applied={T|T < 7333860032099168256}]
I20260927 07:12:51.903007 23327 mvcc.cc:205] Tried to move back new op lower bound from 7333860032099168256 to 7333860032053260288. Current Snapshot: MvccSnapshot[applied={T|T < 7333860032099168256}]
I20260927 07:12:51.904747 22840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:51.904825 22840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:51.905504 22840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 354dbf65c8d74011813fd9407e010bac: Sending DeleteTablet for 3 replicas of tablet 660ffe5bc04440a8b02ab6154caf6d50
I20260927 07:12:51.905804 23233 tablet_service.cc:1514] Processing DeleteTablet for tablet 660ffe5bc04440a8b02ab6154caf6d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:51 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:46318
I20260927 07:12:51.905822 23099 tablet_service.cc:1514] Processing DeleteTablet for tablet 660ffe5bc04440a8b02ab6154caf6d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:51 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:33766
I20260927 07:12:51.905846 22965 tablet_service.cc:1514] Processing DeleteTablet for tablet 660ffe5bc04440a8b02ab6154caf6d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:51 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:59976
I20260927 07:12:51.906028 23335 tablet_replica.cc:330] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: stopping tablet replica
I20260927 07:12:51.906251 23337 tablet_replica.cc:330] T 660ffe5bc04440a8b02ab6154caf6d50 P 8d92ac458e3f45399e12fbf31fe554f5: stopping tablet replica
I20260927 07:12:51.906278 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 22902
I20260927 07:12:51.906229 23335 raft_consensus.cc:2229] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:51.907217 23335 raft_consensus.cc:2258] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:51.907584 23335 ts_tablet_manager.cc:1905] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:51.906141 23336 tablet_replica.cc:330] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: stopping tablet replica
I20260927 07:12:51.907899 23336 raft_consensus.cc:2229] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:51.908054 23336 raft_consensus.cc:2258] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:51.908519 23336 ts_tablet_manager.cc:1905] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:51.908993 23335 ts_tablet_manager.cc:1918] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:51.909060 23335 log.cc:1196] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/660ffe5bc04440a8b02ab6154caf6d50
I20260927 07:12:51.909242 23335 ts_tablet_manager.cc:1939] T 660ffe5bc04440a8b02ab6154caf6d50 P d2f1754c88894881a53c6eabd613b62b: Deleting consensus metadata
I20260927 07:12:51.909621 22827 catalog_manager.cc:4841] TS d2f1754c88894881a53c6eabd613b62b (127.4.174.67:39461): tablet 660ffe5bc04440a8b02ab6154caf6d50 (table test-table [id=dd90f063904946fbb3bb82b44e0bc149]) successfully deleted
I20260927 07:12:51.910378 23336 ts_tablet_manager.cc:1918] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:51.910449 23336 log.cc:1196] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/660ffe5bc04440a8b02ab6154caf6d50
I20260927 07:12:51.910718 23336 ts_tablet_manager.cc:1939] T 660ffe5bc04440a8b02ab6154caf6d50 P cb8ab375bb40410fa9b7639429f1c989: Deleting consensus metadata
I20260927 07:12:51.911106 22826 catalog_manager.cc:4841] TS cb8ab375bb40410fa9b7639429f1c989 (127.4.174.66:33857): tablet 660ffe5bc04440a8b02ab6154caf6d50 (table test-table [id=dd90f063904946fbb3bb82b44e0bc149]) successfully deleted
W20260927 07:12:51.912147 22826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:38545 (error 108)
I20260927 07:12:51.912346 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 23036
I20260927 07:12:51.912880 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:51.912458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:38545 (local address 127.0.0.1:59996)
0927 07:12:51.912813 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:51.912849 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334}
W20260927 07:12:51.913039 22826 catalog_manager.cc:4584] TS 8d92ac458e3f45399e12fbf31fe554f5 (127.4.174.65:38545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 660ffe5bc04440a8b02ab6154caf6d50: Network error: Client connection negotiation failed: client connection to 127.4.174.65:38545: connect: Connection refused (error 111)
I20260927 07:12:51.917155 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 23170
I20260927 07:12:51.921201 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 22808
2026-09-27T07:12:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1745 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-27T07:12:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:51Z Disabled control of system clock
I20260927 07:12:51.946667 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:45255
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:45255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:12:52.025635 23342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:52.025866 23342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:52.025884 23342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:52.027087 23342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:52.027127 23342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:52.027138 23342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:52.027153 23342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:52.027177 23342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:52.028565 23342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:45255
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:45255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:52.028796 23342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:52.029065 23342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:52.033635 23350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:52.033629 23348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:52.033635 23347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:52.033708 23342 server_base.cc:1034] running on GCE node
I20260927 07:12:52.034081 23342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:52.034355 23342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:52.035501 23342 hybrid_clock.cc:648] HybridClock initialized: now 1790493172035476 us; error 39 us; skew 500 ppm
I20260927 07:12:52.036725 23342 webserver.cc:460] Webserver started at http://127.4.174.126:41657/ using document root <none> and password file <none>
I20260927 07:12:52.036947 23342 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:52.036993 23342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:52.037120 23342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:52.038146 23342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "0dbf3b589b584010a1cdc5563149060b"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "b543208090b89c5648f605542360624a"
server_key_iv: "2f78392ce630011dacb0a30bba1718e1"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.038467 23342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "0dbf3b589b584010a1cdc5563149060b"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "b543208090b89c5648f605542360624a"
server_key_iv: "2f78392ce630011dacb0a30bba1718e1"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.039690 23342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:12:52.040380 23356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.040555 23342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:12:52.040678 23342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "0dbf3b589b584010a1cdc5563149060b"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "b543208090b89c5648f605542360624a"
server_key_iv: "2f78392ce630011dacb0a30bba1718e1"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.040786 23342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:52.052501 23342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:52.052814 23342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:52.052947 23342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:52.056658 23342 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:45255
I20260927 07:12:52.056694 23408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:45255 every 8 connection(s)
I20260927 07:12:52.057014 23342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:52.057937 23409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:52.060230 23409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b: Bootstrap starting.
I20260927 07:12:52.060721 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 23342
I20260927 07:12:52.060837 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:52.060878 23409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:52.061058 4793 external_mini_cluster.cc:1428] Setting key 9f690aaaba92b67c62dc2f7e094a4860
I20260927 07:12:52.061203 23409 log.cc:824] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:52.061997 23409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b: No bootstrap required, opened a new log
I20260927 07:12:52.064043 23409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dbf3b589b584010a1cdc5563149060b" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45255 } }
I20260927 07:12:52.064234 23409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:52.064276 23409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dbf3b589b584010a1cdc5563149060b, State: Initialized, Role: FOLLOWER
I20260927 07:12:52.064414 23409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dbf3b589b584010a1cdc5563149060b" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45255 } }
I20260927 07:12:52.064487 23409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:52.064527 23409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:52.064571 23409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:52.065147 23409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dbf3b589b584010a1cdc5563149060b" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45255 } }
I20260927 07:12:52.065248 23409 leader_election.cc:302] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dbf3b589b584010a1cdc5563149060b; no voters:
I20260927 07:12:52.065412 23409 leader_election.cc:288] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:52.065457 23414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:52.065621 23414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [term 1 LEADER]: Becoming Leader. State: Replica: 0dbf3b589b584010a1cdc5563149060b, State: Running, Role: LEADER
I20260927 07:12:52.065670 23409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:52.065716 23414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dbf3b589b584010a1cdc5563149060b" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45255 } }
I20260927 07:12:52.066030 23414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dbf3b589b584010a1cdc5563149060b. Latest consensus state: current_term: 1 leader_uuid: "0dbf3b589b584010a1cdc5563149060b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dbf3b589b584010a1cdc5563149060b" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45255 } } }
I20260927 07:12:52.066099 23414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:52.066185 23415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dbf3b589b584010a1cdc5563149060b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dbf3b589b584010a1cdc5563149060b" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45255 } } }
I20260927 07:12:52.066308 23421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:52.066429 23415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:52.066928 23421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:52.067924 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.061570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.4.174.126:45255)
0927 07:12:52.061755 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.061764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.061800 (+ 36us) server_negotiation.cc:407] Connection header received
0927 07:12:52.062366 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.062376 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.062454 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.062648 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:52.062795 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.064700 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.065513 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.066136 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.066259 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.067180 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:52.067184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:52.067184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:52.067385 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:52.067429 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:52.067444 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:52.067500 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:52.067691 (+ 191us) server_negotiation.cc:299] Negotiation successful
0927 07:12:52.067754 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260927 07:12:52.068605 23421 catalog_manager.cc:1347] Generated new cluster ID: 661eb8d9f9f44911be2be492bf4f8cec
I20260927 07:12:52.068648 23421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:52.091576 23421 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:52.092335 23421 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:52.098501 23421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b: Generated new TSK 0
I20260927 07:12:52.098731 23421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:12:52.101150 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45255
--builtin_ntp_servers=127.4.174.84:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:12:52.195924 23433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:52.196143 23433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:52.196165 23433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:52.197337 23433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:52.197378 23433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:52.197402 23433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:12:52.198815 23433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:52.199049 23433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:52.199331 23433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:52.205056 23438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:52.205096 23441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:52.205367 23433 server_base.cc:1034] running on GCE node
W20260927 07:12:52.205094 23439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:52.205667 23433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:52.205940 23433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:52.207091 23433 hybrid_clock.cc:648] HybridClock initialized: now 1790493172207068 us; error 39 us; skew 500 ppm
I20260927 07:12:52.208547 23433 webserver.cc:460] Webserver started at http://127.4.174.65:40731/ using document root <none> and password file <none>
I20260927 07:12:52.208799 23433 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:52.208856 23433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:52.208979 23433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:52.210289 23433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "dec6680b42452e8fce0954d66980f46a"
server_key_iv: "67084af3f451d6ea6cae2d390b260c81"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.210701 23433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "dec6680b42452e8fce0954d66980f46a"
server_key_iv: "67084af3f451d6ea6cae2d390b260c81"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.212246 23433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:52.212919 23447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.213104 23433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:52.213186 23433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "dec6680b42452e8fce0954d66980f46a"
server_key_iv: "67084af3f451d6ea6cae2d390b260c81"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.213240 23433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:52.233573 23433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:52.233878 23433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:52.233987 23433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:52.234184 23433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:52.234516 23433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:52.234594 23433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.234624 23433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:52.234647 23433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.240175 23433 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:43045
I20260927 07:12:52.240250 23560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:43045 every 8 connection(s)
I20260927 07:12:52.240574 23433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:12:52.245585 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.241295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:50245 (local address 127.4.174.126:45255)
0927 07:12:52.241359 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.241362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.241598 (+ 236us) server_negotiation.cc:407] Connection header received
0927 07:12:52.242098 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.242099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.242111 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.242174 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:52.242882 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.243445 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.244424 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.244657 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.244710 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.245395 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:52.245396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:52.245396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:52.245400 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:52.245413 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:52.245416 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:52.245449 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:52.245508 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 07:12:52.245527 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:12:52.245790 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.241280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45255 (local address 127.4.174.65:50245)
0927 07:12:52.241474 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:52.241516 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:52.241994 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:52.242247 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:52.242250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:52.242259 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:52.242792 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.242797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.243517 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.243518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.244346 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.244349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.244725 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.244726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.244837 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.245282 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:52.245282 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:52.245311 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:52.245333 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:52.245457 (+ 124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:52.245459 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:52.245462 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:52.245504 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 07:12:52.245581 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260927 07:12:52.246237 23561 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45255
I20260927 07:12:52.246323 23561 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:52.246492 23561 heartbeater.cc:511] Master 127.4.174.126:45255 requested a full tablet report, sending...
I20260927 07:12:52.246731 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 23433
I20260927 07:12:52.246810 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:12:52.246994 4793 external_mini_cluster.cc:1428] Setting key f4ec4221686f04a5e4237efc43aade40
I20260927 07:12:52.247016 23373 ts_manager.cc:194] Registered new tserver with Master: 7485f5f8c7e44aa28a27cfc38cf5c66a (127.4.174.65:43045)
I20260927 07:12:52.247995 23373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.65:50245
I20260927 07:12:52.248030 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45255
--builtin_ntp_servers=127.4.174.84:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:12:52.352802 23564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:52.353013 23564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:52.353035 23564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:52.354204 23564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:52.354245 23564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:52.354270 23564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:12:52.355923 23564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:52.356160 23564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:52.356431 23564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:52.361248 23569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:52.361409 23570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:52.361258 23572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:52.361523 23564 server_base.cc:1034] running on GCE node
I20260927 07:12:52.361718 23564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:52.361948 23564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:52.363078 23564 hybrid_clock.cc:648] HybridClock initialized: now 1790493172363068 us; error 26 us; skew 500 ppm
I20260927 07:12:52.364244 23564 webserver.cc:460] Webserver started at http://127.4.174.66:40787/ using document root <none> and password file <none>
I20260927 07:12:52.364460 23564 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:52.364500 23564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:52.364593 23564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:52.365697 23564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "0ee5f1e28eee41c0827dd3caa96bd264"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "65b8189980c4d98aa84d4492f5f746f5"
server_key_iv: "c5910a79d43d346760d5cb3054067821"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.365998 23564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "0ee5f1e28eee41c0827dd3caa96bd264"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "65b8189980c4d98aa84d4492f5f746f5"
server_key_iv: "c5910a79d43d346760d5cb3054067821"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.367237 23564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:12:52.367906 23578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.368055 23564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:12:52.368134 23564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "0ee5f1e28eee41c0827dd3caa96bd264"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "65b8189980c4d98aa84d4492f5f746f5"
server_key_iv: "c5910a79d43d346760d5cb3054067821"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.368206 23564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:52.378353 23564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:52.378687 23564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:52.378814 23564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:52.379029 23564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:52.379333 23564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:52.379365 23564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.379388 23564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:52.379411 23564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.384778 23564 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:36611
I20260927 07:12:52.384831 23691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:36611 every 8 connection(s)
I20260927 07:12:52.385150 23564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:12:52.390094 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.385829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:47581 (local address 127.4.174.126:45255)
0927 07:12:52.385904 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.385906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.386137 (+ 231us) server_negotiation.cc:407] Connection header received
0927 07:12:52.386634 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.386635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.386644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.386702 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:52.387354 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.387840 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.388814 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.389024 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.389081 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.389849 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:52.389850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:52.389850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:52.389856 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:52.389874 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:52.389878 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:52.389935 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:52.389997 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 07:12:52.390025 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:12:52.390297 23694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.385808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45255 (local address 127.4.174.66:47581)
0927 07:12:52.386001 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:52.386056 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:52.386527 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:52.386762 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:52.386765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:52.386775 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:52.387287 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.387293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.387906 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.387906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.388721 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.388724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.389091 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.389092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.389204 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.389688 (+ 484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:52.389689 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:52.389719 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:52.389743 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:52.389937 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:52.389940 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:52.389943 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:52.389990 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 07:12:52.390083 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260927 07:12:52.390808 23692 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45255
I20260927 07:12:52.390902 23692 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:52.391090 23692 heartbeater.cc:511] Master 127.4.174.126:45255 requested a full tablet report, sending...
I20260927 07:12:52.391455 23373 ts_manager.cc:194] Registered new tserver with Master: 0ee5f1e28eee41c0827dd3caa96bd264 (127.4.174.66:36611)
I20260927 07:12:52.391835 23373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.66:47581
I20260927 07:12:52.392568 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 23564
I20260927 07:12:52.392652 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:12:52.392844 4793 external_mini_cluster.cc:1428] Setting key 4f9232b3aaeef3a082676eb8dfdd6cdf
I20260927 07:12:52.393746 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45255
--builtin_ntp_servers=127.4.174.84:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:12:52.473551 23695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:52.473759 23695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:52.473789 23695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:52.474944 23695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:52.474985 23695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:52.475018 23695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:12:52.476357 23695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:45255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:52.476589 23695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:52.476861 23695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:52.481686 23703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:52.481683 23700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:52.481916 23695 server_base.cc:1034] running on GCE node
W20260927 07:12:52.481679 23701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:52.482172 23695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:52.482390 23695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:52.483558 23695 hybrid_clock.cc:648] HybridClock initialized: now 1790493172483531 us; error 40 us; skew 500 ppm
I20260927 07:12:52.484938 23695 webserver.cc:460] Webserver started at http://127.4.174.67:34493/ using document root <none> and password file <none>
I20260927 07:12:52.485179 23695 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:52.485229 23695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:52.485378 23695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:52.486456 23695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "c1b6438ad1ad4829a5a73ce1db22dd63"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "9a842a32405cfdce525df036229395be"
server_key_iv: "871e353fae6844078299a649bda40023"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.486825 23695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "c1b6438ad1ad4829a5a73ce1db22dd63"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "9a842a32405cfdce525df036229395be"
server_key_iv: "871e353fae6844078299a649bda40023"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.488086 23695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:52.488710 23709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.488883 23695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:52.488962 23695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "c1b6438ad1ad4829a5a73ce1db22dd63"
format_stamp: "Formatted at 2026-09-27 07:12:52 on dist-test-slave-zw7p"
server_key: "9a842a32405cfdce525df036229395be"
server_key_iv: "871e353fae6844078299a649bda40023"
server_key_version: "encryptionkey@0"
I20260927 07:12:52.489032 23695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:52.498883 23695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:52.499176 23695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:52.499295 23695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:52.499511 23695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:52.499823 23695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:52.499859 23695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.499887 23695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:52.499910 23695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:52.505187 23695 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:37395
I20260927 07:12:52.505223 23822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:37395 every 8 connection(s)
I20260927 07:12:52.505553 23695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:12:52.508271 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 23695
I20260927 07:12:52.508360 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:12:52.508558 4793 external_mini_cluster.cc:1428] Setting key b0ae00186a76d7e47877da1c08b9bf94
I20260927 07:12:52.510593 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.506213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:43613 (local address 127.4.174.126:45255)
0927 07:12:52.506262 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.506264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.506479 (+ 215us) server_negotiation.cc:407] Connection header received
0927 07:12:52.507068 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.507069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.507082 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.507130 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:52.507823 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.508309 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.509438 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.509624 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.509673 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.510360 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:52.510361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:52.510361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:52.510366 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:52.510380 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:52.510384 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:52.510432 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:52.510491 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 07:12:52.510512 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:12:52.510790 23825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.506189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:45255 (local address 127.4.174.67:43613)
0927 07:12:52.506351 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:52.506385 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:52.506958 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:52.507216 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:52.507221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:52.507229 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:52.507725 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.507730 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.508416 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.508418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.509362 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.509366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.509683 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.509683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.509791 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.510223 (+ 432us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:52.510223 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:52.510250 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:52.510273 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:52.510439 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:52.510441 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:52.510444 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:52.510492 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 07:12:52.510600 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260927 07:12:52.511195 23823 heartbeater.cc:347] Connected to a master server at 127.4.174.126:45255
I20260927 07:12:52.511284 23823 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:52.511427 23823 heartbeater.cc:511] Master 127.4.174.126:45255 requested a full tablet report, sending...
I20260927 07:12:52.511757 23373 ts_manager.cc:194] Registered new tserver with Master: c1b6438ad1ad4829a5a73ce1db22dd63 (127.4.174.67:37395)
I20260927 07:12:52.512084 23373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.67:43613
I20260927 07:12:52.520169 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:12:52.524075 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.521460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.4.174.126:45255)
0927 07:12:52.521513 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.521515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.521609 (+ 94us) server_negotiation.cc:407] Connection header received
0927 07:12:52.521662 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.521662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.521671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.521706 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:52.521822 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.522284 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.523032 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.523234 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.523295 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.523838 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:52.523839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:52.523839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:52.523844 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:52.523857 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:52.523861 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:52.523931 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:52.523986 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 07:12:52.523995 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 07:12:52.525465 23373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:12:52.527578 23373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:12:52.533938 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.529941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37395 (local address 127.0.0.1:39074)
0927 07:12:52.530255 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:52.530267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:52.530337 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:52.530591 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:52.530594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:52.530597 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:52.530669 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.530671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.531733 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.531734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.532876 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.532879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.533648 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.533649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.533723 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.533736 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 07:12:52.533795 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:12:52.533858 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":111,"threads_started":1}
I20260927 07:12:52.533947 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.530135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39074 (local address 127.4.174.67:37395)
0927 07:12:52.530327 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.530332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.530344 (+ 12us) server_negotiation.cc:407] Connection header received
0927 07:12:52.530423 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.530426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.530442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.530737 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:52.530797 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.531659 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.532967 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.533579 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.533636 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.533728 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:52.533821 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 07:12:52.533862 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260927 07:12:52.534464 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.530026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44588 (local address 127.4.174.66:36611)
0927 07:12:52.530184 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.530189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.530371 (+ 182us) server_negotiation.cc:407] Connection header received
0927 07:12:52.530485 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.530487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.530504 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.530574 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:52.530795 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.531910 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.532935 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.533610 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.533672 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.533752 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:52.534347 (+ 595us) server_negotiation.cc:299] Negotiation successful
0927 07:12:52.534387 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260927 07:12:52.534639 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.529940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36611 (local address 127.0.0.1:44588)
0927 07:12:52.530221 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:52.530267 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:52.530397 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:52.530670 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:52.530671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:52.530673 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:52.530719 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.530721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.532003 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.532003 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.532808 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.532810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.533785 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.533786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.533868 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.533872 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:12:52.533921 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 07:12:52.533960 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":37,"thread_start_us":75,"threads_started":1}
I20260927 07:12:52.534780 23757 tablet_service.cc:1467] Processing CreateTablet for tablet 1e429dd6df6d471ba08bab3c8d560e46 (DEFAULT_TABLE table=test-table [id=14526da2068a43a0b5b3c29fc96e2aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:52.534963 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.530155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:43045 (local address 127.0.0.1:39540)
0927 07:12:52.530532 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:52.530541 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:52.530608 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:52.530763 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:52.530763 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:52.530765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:52.530797 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.530798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.532145 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.532146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.533244 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.533246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.534071 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.534072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:52.534148 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.534152 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:12:52.534204 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 07:12:52.534239 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":66,"threads_started":1}
I20260927 07:12:52.535137 23757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e429dd6df6d471ba08bab3c8d560e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:52.535477 23626 tablet_service.cc:1467] Processing CreateTablet for tablet 1e429dd6df6d471ba08bab3c8d560e46 (DEFAULT_TABLE table=test-table [id=14526da2068a43a0b5b3c29fc96e2aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:52.535744 23626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e429dd6df6d471ba08bab3c8d560e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:52.537700 23837 tablet_bootstrap.cc:492] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Bootstrap starting.
I20260927 07:12:52.538044 23836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.530370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39540 (local address 127.4.174.65:43045)
0927 07:12:52.530530 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.530535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.530600 (+ 65us) server_negotiation.cc:407] Connection header received
0927 07:12:52.530669 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.530671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.530684 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.530758 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:52.530854 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.532058 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.533330 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.534009 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:52.534073 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:52.534155 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:52.534217 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 07:12:52.534251 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260927 07:12:52.538220 23838 tablet_bootstrap.cc:492] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Bootstrap starting.
I20260927 07:12:52.538589 23837 tablet_bootstrap.cc:654] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:52.538801 23838 tablet_bootstrap.cc:654] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:52.538703 23495 tablet_service.cc:1467] Processing CreateTablet for tablet 1e429dd6df6d471ba08bab3c8d560e46 (DEFAULT_TABLE table=test-table [id=14526da2068a43a0b5b3c29fc96e2aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:52.538908 23837 log.cc:824] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:52.539021 23495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e429dd6df6d471ba08bab3c8d560e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:52.539106 23838 log.cc:824] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:52.539732 23838 tablet_bootstrap.cc:492] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: No bootstrap required, opened a new log
I20260927 07:12:52.539800 23838 ts_tablet_manager.cc:1397] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:52.541456 23841 tablet_bootstrap.cc:492] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a: Bootstrap starting.
I20260927 07:12:52.541906 23837 tablet_bootstrap.cc:492] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: No bootstrap required, opened a new log
I20260927 07:12:52.541970 23837 ts_tablet_manager.cc:1397] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 07:12:52.542069 23841 tablet_bootstrap.cc:654] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:52.541962 23838 raft_consensus.cc:348] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:52.542124 23838 raft_consensus.cc:374] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:52.542168 23838 raft_consensus.cc:729] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee5f1e28eee41c0827dd3caa96bd264, State: Initialized, Role: FOLLOWER
I20260927 07:12:52.542291 23838 consensus_queue.cc:260] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:52.542394 23841 log.cc:824] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:52.542593 23838 ts_tablet_manager.cc:1428] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:12:52.542706 23692 heartbeater.cc:503] Master 127.4.174.126:45255 was elected leader, sending a full tablet report...
I20260927 07:12:52.544369 23837 raft_consensus.cc:348] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:52.544533 23837 raft_consensus.cc:374] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:52.544574 23837 raft_consensus.cc:729] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b6438ad1ad4829a5a73ce1db22dd63, State: Initialized, Role: FOLLOWER
I20260927 07:12:52.544690 23837 consensus_queue.cc:260] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:52.544952 23837 ts_tablet_manager.cc:1428] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:12:52.544986 23823 heartbeater.cc:503] Master 127.4.174.126:45255 was elected leader, sending a full tablet report...
I20260927 07:12:52.545571 23841 tablet_bootstrap.cc:492] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a: No bootstrap required, opened a new log
I20260927 07:12:52.545629 23841 ts_tablet_manager.cc:1397] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260927 07:12:52.547461 23841 raft_consensus.cc:348] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:52.547582 23841 raft_consensus.cc:374] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:52.547602 23841 raft_consensus.cc:729] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7485f5f8c7e44aa28a27cfc38cf5c66a, State: Initialized, Role: FOLLOWER
I20260927 07:12:52.547734 23841 consensus_queue.cc:260] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:52.547969 23841 ts_tablet_manager.cc:1428] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260927 07:12:52.548029 23561 heartbeater.cc:503] Master 127.4.174.126:45255 was elected leader, sending a full tablet report...
W20260927 07:12:52.635694 23693 tablet.cc:2369] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:12:52.741227 23562 tablet.cc:2369] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:12:52.756090 23824 tablet.cc:2369] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:12:52.997997 23845 raft_consensus.cc:482] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:52.998127 23845 raft_consensus.cc:504] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:52.998481 23845 leader_election.cc:288] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b6438ad1ad4829a5a73ce1db22dd63 (127.4.174.67:37395), 0ee5f1e28eee41c0827dd3caa96bd264 (127.4.174.66:36611)
I20260927 07:12:53.002364 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.998826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:50473 (local address 127.4.174.66:36611)
0927 07:12:52.998877 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.998879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.998889 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:12:52.998982 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.998982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.998992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.999055 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:52.999202 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.999960 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.001478 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:53.002010 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.002087 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:53.002095 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:53.002252 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 07:12:53.002290 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 07:12:53.002364 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.998783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:33893 (local address 127.4.174.67:37395)
0927 07:12:52.998831 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:52.998833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:52.998894 (+ 61us) server_negotiation.cc:407] Connection header received
0927 07:12:52.998982 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:52.998983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:52.998992 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:52.999062 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:52.999203 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:52.999963 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.001472 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:53.002026 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.002087 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:53.002095 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:53.002253 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 07:12:53.002290 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 07:12:53.002388 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.998707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:36611 (local address 127.4.174.65:50473)
0927 07:12:52.998834 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:52.998842 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:52.998909 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:52.999062 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:52.999063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:52.999065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:52.999135 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.999137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:53.000041 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.000042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:53.001380 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:53.001383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:53.002085 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.002086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:53.002182 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:53.002189 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 07:12:53.002241 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 07:12:53.002286 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":17,"thread_start_us":57,"threads_started":1}
I20260927 07:12:53.002405 23846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:52.998700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:37395 (local address 127.4.174.65:33893)
0927 07:12:52.998823 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:52.998834 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:52.998909 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:52.999073 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:52.999074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:52.999075 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:52.999139 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:52.999140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:53.000047 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.000047 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:53.001391 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:53.001393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:53.002089 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.002089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:53.002182 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:53.002190 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:12:53.002241 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 07:12:53.002286 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":124,"thread_start_us":56,"threads_started":1}
I20260927 07:12:53.002745 23646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e429dd6df6d471ba08bab3c8d560e46" candidate_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" is_pre_election: true
I20260927 07:12:53.002899 23646 raft_consensus.cc:2454] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7485f5f8c7e44aa28a27cfc38cf5c66a in term 0.
I20260927 07:12:53.002939 23777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e429dd6df6d471ba08bab3c8d560e46" candidate_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" is_pre_election: true
I20260927 07:12:53.003113 23777 raft_consensus.cc:2454] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7485f5f8c7e44aa28a27cfc38cf5c66a in term 0.
I20260927 07:12:53.003120 23448 leader_election.cc:302] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee5f1e28eee41c0827dd3caa96bd264, 7485f5f8c7e44aa28a27cfc38cf5c66a; no voters:
I20260927 07:12:53.003257 23845 raft_consensus.cc:2781] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:12:53.003304 23845 raft_consensus.cc:482] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:12:53.003331 23845 raft_consensus.cc:3037] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:53.004004 23845 raft_consensus.cc:504] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:53.004119 23845 leader_election.cc:288] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [CANDIDATE]: Term 1 election: Requested vote from peers c1b6438ad1ad4829a5a73ce1db22dd63 (127.4.174.67:37395), 0ee5f1e28eee41c0827dd3caa96bd264 (127.4.174.66:36611)
I20260927 07:12:53.004285 23777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e429dd6df6d471ba08bab3c8d560e46" candidate_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63"
I20260927 07:12:53.004269 23646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e429dd6df6d471ba08bab3c8d560e46" candidate_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee5f1e28eee41c0827dd3caa96bd264"
I20260927 07:12:53.004356 23777 raft_consensus.cc:3037] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:53.004357 23646 raft_consensus.cc:3037] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:53.005209 23777 raft_consensus.cc:2454] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7485f5f8c7e44aa28a27cfc38cf5c66a in term 1.
I20260927 07:12:53.005209 23646 raft_consensus.cc:2454] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7485f5f8c7e44aa28a27cfc38cf5c66a in term 1.
I20260927 07:12:53.005414 23448 leader_election.cc:302] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee5f1e28eee41c0827dd3caa96bd264, 7485f5f8c7e44aa28a27cfc38cf5c66a; no voters:
I20260927 07:12:53.005508 23845 raft_consensus.cc:2781] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:53.005656 23845 raft_consensus.cc:686] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [term 1 LEADER]: Becoming Leader. State: Replica: 7485f5f8c7e44aa28a27cfc38cf5c66a, State: Running, Role: LEADER
I20260927 07:12:53.005743 23845 consensus_queue.cc:237] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } }
I20260927 07:12:53.006534 23373 catalog_manager.cc:5495] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7485f5f8c7e44aa28a27cfc38cf5c66a (127.4.174.65). New cstate: current_term: 1 leader_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7485f5f8c7e44aa28a27cfc38cf5c66a" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 43045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 } health_report { overall_health: UNKNOWN } } }
I20260927 07:12:53.085449 23852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:53.083034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39546 (local address 127.4.174.65:43045)
0927 07:12:53.083138 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:53.083140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:53.083150 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:12:53.083201 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:53.083201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:53.083222 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:53.083285 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:12:53.083410 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:53.083903 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.084622 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:53.084833 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:53.084890 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:53.085001 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:12:53.085096 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:12:53.085165 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:53.085333 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 07:12:53.085358 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
I20260927 07:12:53.087226 23777 raft_consensus.cc:1261] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 1 FOLLOWER]: Refusing update from remote peer 7485f5f8c7e44aa28a27cfc38cf5c66a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:53.087227 23646 raft_consensus.cc:1261] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 1 FOLLOWER]: Refusing update from remote peer 7485f5f8c7e44aa28a27cfc38cf5c66a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:53.087567 23848 consensus_queue.cc:1035] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee5f1e28eee41c0827dd3caa96bd264" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 36611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:53.087699 23845 consensus_queue.cc:1035] T 1e429dd6df6d471ba08bab3c8d560e46 P 7485f5f8c7e44aa28a27cfc38cf5c66a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b6438ad1ad4829a5a73ce1db22dd63" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:53.090143 23856 mvcc.cc:205] Tried to move back new op lower bound from 7333860036962652160 to 7333860036632141824. Current Snapshot: MvccSnapshot[applied={T|T < 7333860036962652160}]
I20260927 07:12:53.090611 23857 mvcc.cc:205] Tried to move back new op lower bound from 7333860036962652160 to 7333860036632141824. Current Snapshot: MvccSnapshot[applied={T|T < 7333860036962652160}]
I20260927 07:12:53.092260 23372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:53.092350 23372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:53.093011 23372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dbf3b589b584010a1cdc5563149060b: Sending DeleteTablet for 3 replicas of tablet 1e429dd6df6d471ba08bab3c8d560e46
I20260927 07:12:53.093308 23626 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e429dd6df6d471ba08bab3c8d560e46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:53 UTC) from {username='slave'} at 127.0.0.1:44588
I20260927 07:12:53.093420 23757 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e429dd6df6d471ba08bab3c8d560e46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:53 UTC) from {username='slave'} at 127.0.0.1:39074
I20260927 07:12:53.093643 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 23433
I20260927 07:12:53.093704 23495 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e429dd6df6d471ba08bab3c8d560e46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:53 UTC) from {username='slave'} at 127.0.0.1:39540
I20260927 07:12:53.094419 23866 tablet_replica.cc:330] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: stopping tablet replica
I20260927 07:12:53.094522 23866 raft_consensus.cc:2229] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:53.094636 23866 raft_consensus.cc:2258] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:53.094991 23866 ts_tablet_manager.cc:1905] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:53.096470 23865 tablet_replica.cc:330] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: stopping tablet replica
I20260927 07:12:53.096589 23865 raft_consensus.cc:2229] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:53.096697 23865 raft_consensus.cc:2258] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:53.096733 23866 ts_tablet_manager.cc:1918] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:53.096803 23866 log.cc:1196] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/1e429dd6df6d471ba08bab3c8d560e46
I20260927 07:12:53.097024 23866 ts_tablet_manager.cc:1939] T 1e429dd6df6d471ba08bab3c8d560e46 P c1b6438ad1ad4829a5a73ce1db22dd63: Deleting consensus metadata
I20260927 07:12:53.097083 23865 ts_tablet_manager.cc:1905] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:53.097398 23358 catalog_manager.cc:4841] TS c1b6438ad1ad4829a5a73ce1db22dd63 (127.4.174.67:37395): tablet 1e429dd6df6d471ba08bab3c8d560e46 (table test-table [id=14526da2068a43a0b5b3c29fc96e2aba]) successfully deleted
I20260927 07:12:53.098459 23865 ts_tablet_manager.cc:1918] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:53.098529 23865 log.cc:1196] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/1e429dd6df6d471ba08bab3c8d560e46
I20260927 07:12:53.098768 23865 ts_tablet_manager.cc:1939] T 1e429dd6df6d471ba08bab3c8d560e46 P 0ee5f1e28eee41c0827dd3caa96bd264: Deleting consensus metadata
I20260927 07:12:53.099442 23357 catalog_manager.cc:4841] TS 0ee5f1e28eee41c0827dd3caa96bd264 (127.4.174.66:36611): tablet 1e429dd6df6d471ba08bab3c8d560e46 (table test-table [id=14526da2068a43a0b5b3c29fc96e2aba]) successfully deleted
W20260927 07:12:53.099956 23358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:43045 (error 108)
I20260927 07:12:53.100435 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:53.100282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:43045 (local address 127.0.0.1:39562)
0927 07:12:53.100364 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:53.100402 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:43045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66,"thread_start_us":43,"threads_started":1}
I20260927 07:12:53.100426 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 23564
W20260927 07:12:53.100979 23358 catalog_manager.cc:4584] TS 7485f5f8c7e44aa28a27cfc38cf5c66a (127.4.174.65:43045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e429dd6df6d471ba08bab3c8d560e46: Network error: Client connection negotiation failed: client connection to 127.4.174.65:43045: connect: Connection refused (error 111)
I20260927 07:12:53.104636 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 23695
I20260927 07:12:53.108911 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 23342
2026-09-27T07:12:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1186 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:12:53 dist-test-slave-zw7p krb5kdc[23871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:12:53 dist-test-slave-zw7p krb5kdc[23871](info): set up 2 sockets
Sep 27 07:12:53 dist-test-slave-zw7p krb5kdc[23871](info): commencing operation
krb5kdc: starting...
W20260927 07:12:54.062129 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:12:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:12:54.173885 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35723
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:35723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:54.252636 23884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:54.252838 23884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:54.252858 23884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:54.253944 23884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:54.253976 23884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:54.253989 23884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:54.254000 23884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:54.254011 23884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:54.255440 23884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:35723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:54.255640 23884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:54.255810 23884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:54.260365 23890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:54.260456 23884 server_base.cc:1034] running on GCE node
W20260927 07:12:54.260363 23892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:54.260387 23889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:54.260844 23884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:54.261125 23884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:54.262269 23884 hybrid_clock.cc:648] HybridClock initialized: now 1790493174262252 us; error 29 us; skew 500 ppm
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:54.264616 23884 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:12:54.265021 23884 webserver.cc:460] Webserver started at http://127.4.174.126:46403/ using document root <none> and password file <none>
I20260927 07:12:54.265220 23884 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:54.265266 23884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:54.265398 23884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:54.266424 23884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "b5402ab3529946cf81a7f6bb635719d6"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "9daef9f9aa7a2719ecf6d9bcc59bd73d"
server_key_iv: "0292068aef83a88f27e7b416a130973d"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.266762 23884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "b5402ab3529946cf81a7f6bb635719d6"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "9daef9f9aa7a2719ecf6d9bcc59bd73d"
server_key_iv: "0292068aef83a88f27e7b416a130973d"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.267952 23884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:54.268646 23899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.268812 23884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:54.268896 23884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "b5402ab3529946cf81a7f6bb635719d6"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "9daef9f9aa7a2719ecf6d9bcc59bd73d"
server_key_iv: "0292068aef83a88f27e7b416a130973d"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.268954 23884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:54.289300 23884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:54.291381 23884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:54.291513 23884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:54.294994 23884 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:35723
I20260927 07:12:54.295037 23951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:35723 every 8 connection(s)
I20260927 07:12:54.295392 23884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:54.296206 23952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:54.298354 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 23884
I20260927 07:12:54.298477 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:54.298534 23952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6: Bootstrap starting.
I20260927 07:12:54.298698 4793 external_mini_cluster.cc:1428] Setting key b784d3d380500d33c6dcf396efb1fd17
I20260927 07:12:54.299190 23952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:54.299463 23952 log.cc:824] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:54.300108 23952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6: No bootstrap required, opened a new log
I20260927 07:12:54.301829 23952 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5402ab3529946cf81a7f6bb635719d6" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35723 } }
I20260927 07:12:54.301964 23952 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:54.301994 23952 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5402ab3529946cf81a7f6bb635719d6, State: Initialized, Role: FOLLOWER
I20260927 07:12:54.302109 23952 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5402ab3529946cf81a7f6bb635719d6" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35723 } }
I20260927 07:12:54.302181 23952 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:54.302211 23952 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:54.302259 23952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:54.302829 23952 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5402ab3529946cf81a7f6bb635719d6" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35723 } }
I20260927 07:12:54.302929 23952 leader_election.cc:302] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5402ab3529946cf81a7f6bb635719d6; no voters:
I20260927 07:12:54.303090 23952 leader_election.cc:288] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:54.303175 23957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:54.303295 23957 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [term 1 LEADER]: Becoming Leader. State: Replica: b5402ab3529946cf81a7f6bb635719d6, State: Running, Role: LEADER
I20260927 07:12:54.303346 23952 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:54.303375 23957 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5402ab3529946cf81a7f6bb635719d6" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35723 } }
I20260927 07:12:54.304505 23959 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5402ab3529946cf81a7f6bb635719d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5402ab3529946cf81a7f6bb635719d6" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35723 } } }
I20260927 07:12:54.304598 23959 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:54.304517 23958 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5402ab3529946cf81a7f6bb635719d6. Latest consensus state: current_term: 1 leader_uuid: "b5402ab3529946cf81a7f6bb635719d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5402ab3529946cf81a7f6bb635719d6" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35723 } } }
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:54.304876 23958 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:54.305217 23968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:54.305626 23968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:54.308909 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.299175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45584 (local address 127.4.174.126:35723)
0927 07:12:54.299380 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.299388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.299412 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:12:54.299954 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.299962 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.300012 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.300137 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:54.300738 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.301628 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.302506 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.303018 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.303119 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.305568 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:54.305572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:54.305573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:54.305584 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:54.307427 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.307759 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.307762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.307762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.307868 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.308041 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.308041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.308042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.308353 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:54.308488 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.308559 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.308696 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260927 07:12:54.311268 23968 catalog_manager.cc:1347] Generated new cluster ID: a0d9fe15bb974f088070e4f29414bfa7
I20260927 07:12:54.311332 23968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:54.317729 23968 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:54.318257 23968 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:54.322494 23968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6: Generated new TSK 0
I20260927 07:12:54.322710 23968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.174.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
Entry for principal kudu/127.4.174.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab.
I20260927 07:12:54.349773 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35723
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:54.437307 23978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:54.437515 23978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:54.437537 23978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:54.438715 23978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:54.438755 23978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:54.438781 23978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:12:54.440212 23978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.65.keytab
--principal=kudu/127.4.174.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:54.440439 23978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:54.440654 23978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 07:12:54.445679 23978 server_base.cc:1034] running on GCE node
W20260927 07:12:54.445640 23986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:54.445649 23984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:54.445649 23983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:54.446003 23978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:54.446249 23978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:54.447381 23978 hybrid_clock.cc:648] HybridClock initialized: now 1790493174447370 us; error 27 us; skew 500 ppm
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:54.449682 23978 init.cc:377] Logged in from keytab as kudu/127.4.174.65@KRBTEST.COM (short username kudu)
I20260927 07:12:54.450093 23978 webserver.cc:460] Webserver started at http://127.4.174.65:43711/ using document root <none> and password file <none>
I20260927 07:12:54.450270 23978 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:54.450309 23978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:54.450392 23978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:54.451465 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "f98e4779be3f47ca80d28a09d4f32104"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "ee9c233578c92882c1f64941664094f3"
server_key_iv: "82d1f048efe208f1f56cfa1e6eb007ef"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.451785 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "f98e4779be3f47ca80d28a09d4f32104"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "ee9c233578c92882c1f64941664094f3"
server_key_iv: "82d1f048efe208f1f56cfa1e6eb007ef"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.452984 23978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:12:54.453630 23993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.453768 23978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:54.453843 23978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "f98e4779be3f47ca80d28a09d4f32104"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "ee9c233578c92882c1f64941664094f3"
server_key_iv: "82d1f048efe208f1f56cfa1e6eb007ef"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.453908 23978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:54.463893 23978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:54.465945 23978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:54.466065 23978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:54.466288 23978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:54.466634 23978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:54.466672 23978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.466703 23978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:54.466727 23978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.472680 23978 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:40171
I20260927 07:12:54.472728 24106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:40171 every 8 connection(s)
I20260927 07:12:54.473033 23978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:12:54.474766 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 23978
I20260927 07:12:54.474884 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:12:54.475102 4793 external_mini_cluster.cc:1428] Setting key c4b6091f52e302a8ebdc636b4c6abed9
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.65@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.174.66@KRBTEST.COM; defaulting to no policy
I20260927 07:12:54.482594 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.473699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:38837 (local address 127.4.174.126:35723)
0927 07:12:54.473745 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.473747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.473974 (+ 227us) server_negotiation.cc:407] Connection header received
0927 07:12:54.474441 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.474442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.474452 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.474513 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:54.475770 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.476349 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.477397 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.477611 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.477674 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.480118 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:54.480119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:54.480119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:54.480127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:54.481319 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.481676 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.481677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.481678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.481729 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.481949 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.481950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.481950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.482059 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:54.482114 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.482384 (+ 270us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.482468 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:12:54.482754 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.473672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35723 (local address 127.4.174.65:38837)
0927 07:12:54.473830 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.473872 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:54.474359 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:54.474619 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:54.474624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:54.475382 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:54.475646 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.475654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.476459 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.476460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.477302 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.477305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.477690 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.477691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.477831 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.478336 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:54.478344 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:54.480030 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:54.481392 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:54.481399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:54.481401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:54.481611 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:54.481784 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:54.481785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:54.481785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:54.481895 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:54.482112 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:54.482119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:54.482258 (+ 139us) client_negotiation.cc:769] Sending connection context
0927 07:12:54.482354 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 07:12:54.482513 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260927 07:12:54.483222 24107 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35723
I20260927 07:12:54.483320 24107 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:54.483508 24107 heartbeater.cc:511] Master 127.4.174.126:35723 requested a full tablet report, sending...
I20260927 07:12:54.483999 23916 ts_manager.cc:194] Registered new tserver with Master: f98e4779be3f47ca80d28a09d4f32104 (127.4.174.65:40171)
I20260927 07:12:54.484715 23916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.65@KRBTEST.COM'} at 127.4.174.65:38837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
Entry for principal kudu/127.4.174.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab.
I20260927 07:12:54.499140 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35723
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:54.587459 24112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:54.587742 24112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:54.587823 24112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:54.589581 24112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:54.589635 24112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:54.589687 24112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:12:54.591861 24112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.66.keytab
--principal=kudu/127.4.174.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:54.592120 24112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:54.592398 24112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:54.598242 24118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:54.598251 24117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:54.598241 24120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:54.598472 24112 server_base.cc:1034] running on GCE node
I20260927 07:12:54.598687 24112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:54.598943 24112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:54.600103 24112 hybrid_clock.cc:648] HybridClock initialized: now 1790493174600081 us; error 34 us; skew 500 ppm
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:54.603149 24112 init.cc:377] Logged in from keytab as kudu/127.4.174.66@KRBTEST.COM (short username kudu)
I20260927 07:12:54.603719 24112 webserver.cc:460] Webserver started at http://127.4.174.66:37845/ using document root <none> and password file <none>
I20260927 07:12:54.603966 24112 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:54.604027 24112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:54.604154 24112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:54.605527 24112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "406ffe4a6a6d424683836895495a5629"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "4df5aaaab30182bda9d88bf027e844f4"
server_key_iv: "baa670137b66fd441450b012de3f8891"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.606002 24112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "406ffe4a6a6d424683836895495a5629"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "4df5aaaab30182bda9d88bf027e844f4"
server_key_iv: "baa670137b66fd441450b012de3f8891"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.607671 24112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260927 07:12:54.608435 24127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.608592 24112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.608675 24112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "406ffe4a6a6d424683836895495a5629"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "4df5aaaab30182bda9d88bf027e844f4"
server_key_iv: "baa670137b66fd441450b012de3f8891"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.608744 24112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:54.629967 24112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:54.631949 24112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:54.632085 24112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:54.632301 24112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:54.632596 24112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:54.632628 24112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.632659 24112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:54.632676 24112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.638000 24112 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:41895
I20260927 07:12:54.638051 24240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:41895 every 8 connection(s)
I20260927 07:12:54.638350 24112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:12:54.643843 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 24112
I20260927 07:12:54.643931 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.66@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:54.644109 4793 external_mini_cluster.cc:1428] Setting key 67df8080992ba89783f2a1da0dc26ede
I20260927 07:12:54.647007 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.639097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:36337 (local address 127.4.174.126:35723)
0927 07:12:54.639189 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.639191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.639429 (+ 238us) server_negotiation.cc:407] Connection header received
0927 07:12:54.639903 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.639904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.639914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.639971 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:54.640844 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.641332 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.642260 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.642459 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.642514 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.644559 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:54.644560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:54.644560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:54.644567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:54.645788 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.646139 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.646141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.646141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.646191 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.646407 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.646408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.646408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.646516 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:54.646583 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.646841 (+ 258us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.646920 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 07:12:54.647168 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.639063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35723 (local address 127.4.174.66:36337)
0927 07:12:54.639309 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.639346 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:54.639821 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:54.640029 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:54.640032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:54.640583 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:54.640776 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.640781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.641396 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.641396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.642179 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.642181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.642535 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.642536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.642673 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.643062 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:54.643070 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:54.644475 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:54.645862 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:54.645869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:54.645871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:54.646078 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:54.646242 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:54.646242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:54.646243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:54.646350 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:54.646580 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:54.646587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:54.646720 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 07:12:54.646814 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 07:12:54.646963 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":171,"threads_started":1}
I20260927 07:12:54.647553 24241 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35723
I20260927 07:12:54.647660 24241 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:54.647850 24241 heartbeater.cc:511] Master 127.4.174.126:35723 requested a full tablet report, sending...
I20260927 07:12:54.648216 23916 ts_manager.cc:194] Registered new tserver with Master: 406ffe4a6a6d424683836895495a5629 (127.4.174.66:41895)
I20260927 07:12:54.648582 23916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.66@KRBTEST.COM'} at 127.4.174.66:36337
WARNING: no policy specified for kudu/127.4.174.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
Entry for principal kudu/127.4.174.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab.
I20260927 07:12:54.667521 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35723
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:54.746167 24246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:54.746379 24246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:54.746409 24246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:54.747617 24246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:54.747665 24246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:54.747701 24246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:12:54.749167 24246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.67.keytab
--principal=kudu/127.4.174.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:54.749414 24246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:54.749639 24246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:54.754544 24251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:54.754544 24254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:54.754534 24252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:54.754750 24246 server_base.cc:1034] running on GCE node
I20260927 07:12:54.754931 24246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:54.755132 24246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:54.756271 24246 hybrid_clock.cc:648] HybridClock initialized: now 1790493174756260 us; error 29 us; skew 500 ppm
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:54.758656 24246 init.cc:377] Logged in from keytab as kudu/127.4.174.67@KRBTEST.COM (short username kudu)
I20260927 07:12:54.759071 24246 webserver.cc:460] Webserver started at http://127.4.174.67:39511/ using document root <none> and password file <none>
I20260927 07:12:54.759262 24246 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:54.759300 24246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:54.759389 24246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:54.760373 24246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "1251055870ed4fbd869ed9179f8d4494"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "f23890af3470396bb8f8055aa84fc727"
server_key_iv: "29aa42068c656a2e913d35d0e641a32f"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.760670 24246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "1251055870ed4fbd869ed9179f8d4494"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "f23890af3470396bb8f8055aa84fc727"
server_key_iv: "29aa42068c656a2e913d35d0e641a32f"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.761859 24246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 07:12:54.762490 24261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.762640 24246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:54.762717 24246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "1251055870ed4fbd869ed9179f8d4494"
format_stamp: "Formatted at 2026-09-27 07:12:54 on dist-test-slave-zw7p"
server_key: "f23890af3470396bb8f8055aa84fc727"
server_key_iv: "29aa42068c656a2e913d35d0e641a32f"
server_key_version: "encryptionkey@0"
I20260927 07:12:54.762769 24246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:54.778825 24246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:54.780865 24246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:54.780973 24246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:54.781154 24246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:54.781416 24246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:54.781446 24246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.781466 24246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:54.781482 24246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:54.786895 24246 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:35243
I20260927 07:12:54.786965 24374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:35243 every 8 connection(s)
I20260927 07:12:54.787246 24246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:12:54.792176 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 24246
I20260927 07:12:54.792264 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:12:54.792454 4793 external_mini_cluster.cc:1428] Setting key d812ba851e5a134192d22f708265ed0d
Sep 27 07:12:54 dist-test-slave-zw7p krb5kdc[23871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.67@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:12:54.796092 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.787917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:57179 (local address 127.4.174.126:35723)
0927 07:12:54.787975 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.787977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.788161 (+ 184us) server_negotiation.cc:407] Connection header received
0927 07:12:54.788672 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.788673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.788686 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.788733 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:54.789681 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.790178 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.791183 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.791380 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.791435 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.793518 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:54.793519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:54.793520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:54.793527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:54.794845 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.795213 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.795214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.795215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.795269 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.795482 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.795483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.795483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.795590 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:54.795645 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.795905 (+ 260us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.796004 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 07:12:54.796267 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.787900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:35723 (local address 127.4.174.67:57179)
0927 07:12:54.788043 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.788083 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:54.788577 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:54.788806 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:54.788811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:54.789371 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:54.789585 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.789590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.790258 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.790259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.791102 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.791104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.791447 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.791448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.791574 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.792040 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:12:54.792048 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:54.793432 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:54.794914 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:54.794921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:54.794923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:54.795139 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:54.795322 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:12:54.795322 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:12:54.795323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:12:54.795427 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:12:54.795643 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:54.795650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:54.795789 (+ 139us) client_negotiation.cc:769] Sending connection context
0927 07:12:54.795877 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 07:12:54.796048 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
I20260927 07:12:54.796669 24375 heartbeater.cc:347] Connected to a master server at 127.4.174.126:35723
I20260927 07:12:54.796765 24375 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:54.796937 24375 heartbeater.cc:511] Master 127.4.174.126:35723 requested a full tablet report, sending...
I20260927 07:12:54.797319 23916 ts_manager.cc:194] Registered new tserver with Master: 1251055870ed4fbd869ed9179f8d4494 (127.4.174.67:35243)
I20260927 07:12:54.797674 23916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.174.67@KRBTEST.COM'} at 127.4.174.67:57179
I20260927 07:12:54.804028 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:12:54.811076 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.805476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45586 (local address 127.4.174.126:35723)
0927 07:12:54.805526 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.805528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.805658 (+ 130us) server_negotiation.cc:407] Connection header received
0927 07:12:54.805721 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.805722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.805733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.805777 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:54.806286 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.806832 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.807518 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.807710 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.807762 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.808823 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:54.808825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:54.808825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:54.808832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:54.810054 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.810340 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.810341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.810341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.810396 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:54.810584 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:54.810585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:54.810585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:54.810727 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:54.810812 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.810910 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.810989 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 07:12:54.812510 23916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:12:54.814015 23916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:12:54.820171 24383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.815969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:35243 (local address 127.0.0.1:53396)
0927 07:12:54.816205 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.816215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:54.816264 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:54.816655 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:54.816658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:54.816660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:54.816720 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.816722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.818059 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.818060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.819260 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.819262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.819880 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.819881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.819962 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.819976 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 07:12:54.820046 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 07:12:54.820098 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":8,"thread_start_us":62,"threads_started":1}
I20260927 07:12:54.820207 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.816320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53396 (local address 127.4.174.67:35243)
0927 07:12:54.816467 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.816471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.816490 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:12:54.816536 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.816538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.816555 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.816639 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:54.816804 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.817962 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.819332 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.819802 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.819869 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.819931 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.820082 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.820130 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260927 07:12:54.821007 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.815971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:41895 (local address 127.0.0.1:35390)
0927 07:12:54.816722 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.816732 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:54.816776 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:54.817031 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:54.817031 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:54.817033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:54.817069 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.817070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.817895 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.817896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.819260 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.819262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.820779 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.820780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.820836 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.820840 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:12:54.820893 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 07:12:54.820930 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":31,"thread_start_us":67,"threads_started":1}
I20260927 07:12:54.821053 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.816201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.4.174.66:41895)
0927 07:12:54.816387 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.816391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.816826 (+ 435us) server_negotiation.cc:407] Connection header received
0927 07:12:54.816869 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.816872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.816888 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.816953 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:54.817132 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.817808 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.819977 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.820630 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.820688 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.820743 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.820923 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.820967 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
I20260927 07:12:54.821305 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.815961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:40171 (local address 127.0.0.1:38424)
0927 07:12:54.816130 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.816167 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:54.816227 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:54.816747 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:54.816748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:54.816750 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:54.816791 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.816792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.819356 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.819356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.820046 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.820048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.821002 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.821003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.821149 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.821154 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:12:54.821197 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 07:12:54.821235 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260927 07:12:54.821208 24309 tablet_service.cc:1467] Processing CreateTablet for tablet f067c2415de94c4aa6ac44759cdbb32c (DEFAULT_TABLE table=test-table [id=c4ecdaf0af7844b3afdad66a46ed8e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:54.821348 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.816147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.4.174.65:40171)
0927 07:12:54.816498 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.816502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.816511 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:54.816543 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.816545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.816558 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.816639 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:54.816868 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.817954 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.820136 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.820855 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.820916 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.820969 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.821233 (+ 264us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.821277 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":81,"threads_started":1}
I20260927 07:12:54.821556 24309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f067c2415de94c4aa6ac44759cdbb32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:54.821892 24041 tablet_service.cc:1467] Processing CreateTablet for tablet f067c2415de94c4aa6ac44759cdbb32c (DEFAULT_TABLE table=test-table [id=c4ecdaf0af7844b3afdad66a46ed8e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:54.822139 24041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f067c2415de94c4aa6ac44759cdbb32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:54.823020 24175 tablet_service.cc:1467] Processing CreateTablet for tablet f067c2415de94c4aa6ac44759cdbb32c (DEFAULT_TABLE table=test-table [id=c4ecdaf0af7844b3afdad66a46ed8e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:54.823321 24175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f067c2415de94c4aa6ac44759cdbb32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:54.824436 24389 tablet_bootstrap.cc:492] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: Bootstrap starting.
I20260927 07:12:54.825078 24389 tablet_bootstrap.cc:654] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:54.825464 24389 log.cc:824] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:54.825739 24390 tablet_bootstrap.cc:492] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104: Bootstrap starting.
I20260927 07:12:54.825843 24392 tablet_bootstrap.cc:492] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Bootstrap starting.
I20260927 07:12:54.826151 24389 tablet_bootstrap.cc:492] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: No bootstrap required, opened a new log
I20260927 07:12:54.826207 24389 ts_tablet_manager.cc:1397] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:54.826398 24390 tablet_bootstrap.cc:654] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:54.826851 24392 tablet_bootstrap.cc:654] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:54.826937 24390 log.cc:824] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:54.827342 24392 log.cc:824] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:54.827749 24390 tablet_bootstrap.cc:492] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104: No bootstrap required, opened a new log
I20260927 07:12:54.827819 24390 ts_tablet_manager.cc:1397] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 07:12:54.827971 24392 tablet_bootstrap.cc:492] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: No bootstrap required, opened a new log
I20260927 07:12:54.828027 24392 ts_tablet_manager.cc:1397] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:54.828107 24389 raft_consensus.cc:348] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.828282 24389 raft_consensus.cc:374] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:54.828320 24389 raft_consensus.cc:729] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1251055870ed4fbd869ed9179f8d4494, State: Initialized, Role: FOLLOWER
I20260927 07:12:54.828459 24389 consensus_queue.cc:260] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.828723 24389 ts_tablet_manager.cc:1428] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 07:12:54.828862 24375 heartbeater.cc:503] Master 127.4.174.126:35723 was elected leader, sending a full tablet report...
I20260927 07:12:54.829662 24392 raft_consensus.cc:348] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.829811 24392 raft_consensus.cc:374] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:54.829834 24392 raft_consensus.cc:729] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406ffe4a6a6d424683836895495a5629, State: Initialized, Role: FOLLOWER
I20260927 07:12:54.829942 24392 consensus_queue.cc:260] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.829982 24390 raft_consensus.cc:348] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.830142 24390 raft_consensus.cc:374] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:54.830166 24241 heartbeater.cc:503] Master 127.4.174.126:35723 was elected leader, sending a full tablet report...
I20260927 07:12:54.830180 24390 raft_consensus.cc:729] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f98e4779be3f47ca80d28a09d4f32104, State: Initialized, Role: FOLLOWER
I20260927 07:12:54.830174 24392 ts_tablet_manager.cc:1428] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:12:54.830284 24390 consensus_queue.cc:260] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.830541 24107 heartbeater.cc:503] Master 127.4.174.126:35723 was elected leader, sending a full tablet report...
I20260927 07:12:54.830535 24390 ts_tablet_manager.cc:1428] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 07:12:54.880848 24397 raft_consensus.cc:482] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:54.880966 24397 raft_consensus.cc:504] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.881378 24397 leader_election.cc:288] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1251055870ed4fbd869ed9179f8d4494 (127.4.174.67:35243), 406ffe4a6a6d424683836895495a5629 (127.4.174.66:41895)
I20260927 07:12:54.884081 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.881562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:41895 (local address 127.4.174.65:50635)
0927 07:12:54.881607 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.881620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:54.881661 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:54.881816 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:54.881817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:54.881819 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:54.881890 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.881892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.882509 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.882510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.883300 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.883302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.883831 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.883832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.883913 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.883915 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:12:54.883961 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 07:12:54.884010 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260927 07:12:54.884080 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.881609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:50635 (local address 127.4.174.66:41895)
0927 07:12:54.881661 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.881663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.881673 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:12:54.881728 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.881729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.881738 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.881790 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:54.881969 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.882433 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.883388 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.883764 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.883824 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.883833 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.883977 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.884014 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:12:54.884183 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.881641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:47007 (local address 127.4.174.67:35243)
0927 07:12:54.881707 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.881709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.881927 (+ 218us) server_negotiation.cc:407] Connection header received
0927 07:12:54.881973 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.881974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.881985 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.882024 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:54.882156 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.882635 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.883520 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.883892 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.883953 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.883961 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.884091 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.884123 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 07:12:54.884302 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.881563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:35243 (local address 127.4.174.65:47007)
0927 07:12:54.881869 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.881877 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:54.881922 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:54.882035 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:54.882036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:54.882037 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:54.882106 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.882107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.882704 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.882705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.883447 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:54.883449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.883966 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.883967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:54.884036 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.884039 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:54.884081 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 07:12:54.884116 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":150,"thread_start_us":94,"threads_started":1}
I20260927 07:12:54.884648 24329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f067c2415de94c4aa6ac44759cdbb32c" candidate_uuid: "f98e4779be3f47ca80d28a09d4f32104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1251055870ed4fbd869ed9179f8d4494" is_pre_election: true
I20260927 07:12:54.884653 24195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f067c2415de94c4aa6ac44759cdbb32c" candidate_uuid: "f98e4779be3f47ca80d28a09d4f32104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406ffe4a6a6d424683836895495a5629" is_pre_election: true
I20260927 07:12:54.884806 24329 raft_consensus.cc:2454] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f98e4779be3f47ca80d28a09d4f32104 in term 0.
I20260927 07:12:54.884806 24195 raft_consensus.cc:2454] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f98e4779be3f47ca80d28a09d4f32104 in term 0.
I20260927 07:12:54.885011 23994 leader_election.cc:302] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406ffe4a6a6d424683836895495a5629, f98e4779be3f47ca80d28a09d4f32104; no voters:
I20260927 07:12:54.885144 24397 raft_consensus.cc:2781] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:12:54.885217 24397 raft_consensus.cc:482] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:12:54.885234 24397 raft_consensus.cc:3037] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:54.885913 24397 raft_consensus.cc:504] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.886045 24397 leader_election.cc:288] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [CANDIDATE]: Term 1 election: Requested vote from peers 1251055870ed4fbd869ed9179f8d4494 (127.4.174.67:35243), 406ffe4a6a6d424683836895495a5629 (127.4.174.66:41895)
I20260927 07:12:54.886206 24195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f067c2415de94c4aa6ac44759cdbb32c" candidate_uuid: "f98e4779be3f47ca80d28a09d4f32104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406ffe4a6a6d424683836895495a5629"
I20260927 07:12:54.886251 24329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f067c2415de94c4aa6ac44759cdbb32c" candidate_uuid: "f98e4779be3f47ca80d28a09d4f32104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1251055870ed4fbd869ed9179f8d4494"
I20260927 07:12:54.886327 24329 raft_consensus.cc:3037] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:54.886327 24195 raft_consensus.cc:3037] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:54.887253 24329 raft_consensus.cc:2454] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f98e4779be3f47ca80d28a09d4f32104 in term 1.
I20260927 07:12:54.887253 24195 raft_consensus.cc:2454] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f98e4779be3f47ca80d28a09d4f32104 in term 1.
I20260927 07:12:54.887426 23996 leader_election.cc:302] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1251055870ed4fbd869ed9179f8d4494, f98e4779be3f47ca80d28a09d4f32104; no voters:
I20260927 07:12:54.887517 24397 raft_consensus.cc:2781] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:54.887666 24397 raft_consensus.cc:686] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 1 LEADER]: Becoming Leader. State: Replica: f98e4779be3f47ca80d28a09d4f32104, State: Running, Role: LEADER
I20260927 07:12:54.887755 24397 consensus_queue.cc:237] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } }
I20260927 07:12:54.888576 23916 catalog_manager.cc:5495] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 reported cstate change: term changed from 0 to 1, leader changed from <none> to f98e4779be3f47ca80d28a09d4f32104 (127.4.174.65). New cstate: current_term: 1 leader_uuid: "f98e4779be3f47ca80d28a09d4f32104" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98e4779be3f47ca80d28a09d4f32104" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 40171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 } health_report { overall_health: UNKNOWN } } }
W20260927 07:12:54.888880 24242 tablet.cc:2369] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:12:54.897441 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.895326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.4.174.65:40171)
0927 07:12:54.895366 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:54.895367 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:54.895392 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:12:54.895423 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:54.895424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:54.895439 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:54.895497 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:12:54.895591 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.896071 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.896803 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:54.896974 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:54.897026 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:54.897197 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:12:54.897296 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:12:54.897351 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:54.897366 (+ 15us) server_negotiation.cc:299] Negotiation successful
0927 07:12:54.897384 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:12:54.899151 24329 raft_consensus.cc:1261] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 1 FOLLOWER]: Refusing update from remote peer f98e4779be3f47ca80d28a09d4f32104: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:54.899178 24195 raft_consensus.cc:1261] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 1 FOLLOWER]: Refusing update from remote peer f98e4779be3f47ca80d28a09d4f32104: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:54.899478 24399 consensus_queue.cc:1035] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1251055870ed4fbd869ed9179f8d4494" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:54.899616 24397 consensus_queue.cc:1035] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406ffe4a6a6d424683836895495a5629" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 41895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:54.901860 24402 mvcc.cc:205] Tried to move back new op lower bound from 7333860044384440320 to 7333860044340862976. Current Snapshot: MvccSnapshot[applied={T|T < 7333860044384440320}]
I20260927 07:12:54.902037 24403 mvcc.cc:205] Tried to move back new op lower bound from 7333860044384440320 to 7333860044340862976. Current Snapshot: MvccSnapshot[applied={T|T < 7333860044384440320}]
I20260927 07:12:54.903105 24404 mvcc.cc:205] Tried to move back new op lower bound from 7333860044384440320 to 7333860044340862976. Current Snapshot: MvccSnapshot[applied={T|T < 7333860044384440320 or (T in {7333860044384440320})}]
I20260927 07:12:54.904253 23915 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:54.904352 23915 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:54.905078 23915 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5402ab3529946cf81a7f6bb635719d6: Sending DeleteTablet for 3 replicas of tablet f067c2415de94c4aa6ac44759cdbb32c
I20260927 07:12:54.905406 24041 tablet_service.cc:1514] Processing DeleteTablet for tablet f067c2415de94c4aa6ac44759cdbb32c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:54 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:38424
I20260927 07:12:54.905408 24175 tablet_service.cc:1514] Processing DeleteTablet for tablet f067c2415de94c4aa6ac44759cdbb32c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:54 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:35390
I20260927 07:12:54.905426 24309 tablet_service.cc:1514] Processing DeleteTablet for tablet f067c2415de94c4aa6ac44759cdbb32c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:54 UTC) from {username='kudu', principal='kudu/127.4.174.126@KRBTEST.COM'} at 127.0.0.1:53396
I20260927 07:12:54.905645 24413 tablet_replica.cc:330] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104: stopping tablet replica
I20260927 07:12:54.905741 24413 raft_consensus.cc:2229] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:12:54.905833 24413 raft_consensus.cc:2258] T f067c2415de94c4aa6ac44759cdbb32c P f98e4779be3f47ca80d28a09d4f32104 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:54.905961 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 23978
I20260927 07:12:54.905987 24414 tablet_replica.cc:330] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: stopping tablet replica
I20260927 07:12:54.906582 24414 raft_consensus.cc:2229] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:54.906693 24414 raft_consensus.cc:2258] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:54.906831 24412 tablet_replica.cc:330] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: stopping tablet replica
I20260927 07:12:54.906914 24412 raft_consensus.cc:2229] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:54.906992 24412 raft_consensus.cc:2258] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:54.907214 24414 ts_tablet_manager.cc:1905] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:54.907380 24412 ts_tablet_manager.cc:1905] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:54.908701 24414 ts_tablet_manager.cc:1918] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:54.908762 24414 log.cc:1196] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/f067c2415de94c4aa6ac44759cdbb32c
I20260927 07:12:54.908902 24412 ts_tablet_manager.cc:1918] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:54.908972 24412 log.cc:1196] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/f067c2415de94c4aa6ac44759cdbb32c
I20260927 07:12:54.909027 24414 ts_tablet_manager.cc:1939] T f067c2415de94c4aa6ac44759cdbb32c P 1251055870ed4fbd869ed9179f8d4494: Deleting consensus metadata
I20260927 07:12:54.909209 24412 ts_tablet_manager.cc:1939] T f067c2415de94c4aa6ac44759cdbb32c P 406ffe4a6a6d424683836895495a5629: Deleting consensus metadata
I20260927 07:12:54.909371 23902 catalog_manager.cc:4841] TS 1251055870ed4fbd869ed9179f8d4494 (127.4.174.67:35243): tablet f067c2415de94c4aa6ac44759cdbb32c (table test-table [id=c4ecdaf0af7844b3afdad66a46ed8e2d]) successfully deleted
I20260927 07:12:54.909734 23900 catalog_manager.cc:4841] TS 406ffe4a6a6d424683836895495a5629 (127.4.174.66:41895): tablet f067c2415de94c4aa6ac44759cdbb32c (table test-table [id=c4ecdaf0af7844b3afdad66a46ed8e2d]) successfully deleted
W20260927 07:12:54.911909 23901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:40171 (error 108)
I20260927 07:12:54.912104 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 24112
I20260927 07:12:54.912371 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:54.912276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:40171 (local address 127.0.0.1:38438)
0927 07:12:54.912318 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:54.912343 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:40171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 07:12:54.912580 23901 catalog_manager.cc:4584] TS f98e4779be3f47ca80d28a09d4f32104 (127.4.174.65:40171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f067c2415de94c4aa6ac44759cdbb32c: Network error: Client connection negotiation failed: client connection to 127.4.174.65:40171: connect: Connection refused (error 111)
I20260927 07:12:54.916837 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 24246
I20260927 07:12:54.922919 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 23884
2026-09-27T07:12:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1815 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-27T07:12:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:54Z Disabled control of system clock
I20260927 07:12:54.949257 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40337
--webserver_interface=127.4.174.126
--webserver_port=0
--builtin_ntp_servers=127.4.174.84:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:40337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:12:55.028272 24419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:55.028504 24419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:55.028537 24419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:55.029938 24419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:55.029978 24419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:55.029989 24419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:55.029999 24419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:55.030010 24419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:55.031312 24419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:40337
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:40337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:55.031498 24419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:55.031723 24419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:55.036229 24425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:55.036193 24427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:55.036201 24424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:55.036252 24419 server_base.cc:1034] running on GCE node
I20260927 07:12:55.036626 24419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:55.036906 24419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:55.038040 24419 hybrid_clock.cc:648] HybridClock initialized: now 1790493175038023 us; error 32 us; skew 500 ppm
I20260927 07:12:55.039207 24419 webserver.cc:460] Webserver started at http://127.4.174.126:35925/ using document root <none> and password file <none>
I20260927 07:12:55.039433 24419 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:55.039477 24419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:55.039602 24419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:55.040649 24419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "5cef726e8a4d432d9cb17df3f8207870"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "bd2a9d5268c84414b85cf282e22ca6f6"
server_key_iv: "62af3615ff533b0864f5c59499dfe1d7"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.040969 24419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "5cef726e8a4d432d9cb17df3f8207870"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "bd2a9d5268c84414b85cf282e22ca6f6"
server_key_iv: "62af3615ff533b0864f5c59499dfe1d7"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.042328 24419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:12:55.043162 24433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.043350 24419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.043437 24419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "5cef726e8a4d432d9cb17df3f8207870"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "bd2a9d5268c84414b85cf282e22ca6f6"
server_key_iv: "62af3615ff533b0864f5c59499dfe1d7"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.043516 24419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:55.052482 24419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:55.052775 24419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:55.052906 24419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:55.056602 24419 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:40337
I20260927 07:12:55.056630 24485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:40337 every 8 connection(s)
I20260927 07:12:55.056945 24419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:55.057857 24486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:55.060101 24486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870: Bootstrap starting.
I20260927 07:12:55.060606 24486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:55.060833 24486 log.cc:824] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:55.061425 24486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870: No bootstrap required, opened a new log
I20260927 07:12:55.063066 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 24419
I20260927 07:12:55.063176 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:55.063089 24486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cef726e8a4d432d9cb17df3f8207870" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40337 } }
I20260927 07:12:55.063259 24486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:55.063299 24486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cef726e8a4d432d9cb17df3f8207870, State: Initialized, Role: FOLLOWER
I20260927 07:12:55.063359 4793 external_mini_cluster.cc:1428] Setting key 9700b77842e26e3e9276d8a8c8068cdc
I20260927 07:12:55.063444 24486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cef726e8a4d432d9cb17df3f8207870" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40337 } }
I20260927 07:12:55.063525 24486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:55.063562 24486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:55.063602 24486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:55.064215 24486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cef726e8a4d432d9cb17df3f8207870" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40337 } }
I20260927 07:12:55.064313 24486 leader_election.cc:302] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cef726e8a4d432d9cb17df3f8207870; no voters:
I20260927 07:12:55.064460 24486 leader_election.cc:288] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:55.064555 24491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:55.064703 24491 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [term 1 LEADER]: Becoming Leader. State: Replica: 5cef726e8a4d432d9cb17df3f8207870, State: Running, Role: LEADER
I20260927 07:12:55.064728 24486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:55.064859 24491 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cef726e8a4d432d9cb17df3f8207870" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40337 } }
I20260927 07:12:55.065231 24492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cef726e8a4d432d9cb17df3f8207870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cef726e8a4d432d9cb17df3f8207870" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40337 } } }
I20260927 07:12:55.065318 24492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:55.065461 24493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cef726e8a4d432d9cb17df3f8207870. Latest consensus state: current_term: 1 leader_uuid: "5cef726e8a4d432d9cb17df3f8207870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cef726e8a4d432d9cb17df3f8207870" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 40337 } } }
I20260927 07:12:55.065521 24493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:55.065622 24500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:55.066324 24500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:55.067935 24500 catalog_manager.cc:1347] Generated new cluster ID: c37b81706eaa4c6cbdfa7d2fe00093b7
I20260927 07:12:55.067992 24500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:55.069037 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.063831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56954 (local address 127.4.174.126:40337)
0927 07:12:55.064025 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.064033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.064058 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:12:55.064153 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.064158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.064216 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.064337 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:55.064538 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.066134 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.067212 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.067697 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.067807 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.068415 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:55.068419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:55.068420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:55.068589 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:55.068621 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:55.068631 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:55.068683 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.068758 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.068836 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260927 07:12:55.095461 24500 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:55.096024 24500 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:55.100010 24500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870: Generated new TSK 0
I20260927 07:12:55.100180 24500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:12:55.105670 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.65:0
--local_ip_for_outbound_sockets=127.4.174.65
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40337
--builtin_ntp_servers=127.4.174.84:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:12:55.207412 24510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:55.207603 24510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:55.207623 24510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:55.208724 24510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:55.208760 24510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:55.208786 24510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.65
I20260927 07:12:55.210137 24510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.174.65
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.65
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:55.210357 24510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:55.210657 24510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:55.215242 24515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:55.215270 24518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:55.215400 24516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:55.215517 24510 server_base.cc:1034] running on GCE node
I20260927 07:12:55.215703 24510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:55.215922 24510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:55.217064 24510 hybrid_clock.cc:648] HybridClock initialized: now 1790493175217052 us; error 28 us; skew 500 ppm
I20260927 07:12:55.218243 24510 webserver.cc:460] Webserver started at http://127.4.174.65:39871/ using document root <none> and password file <none>
I20260927 07:12:55.218461 24510 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:55.218508 24510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:55.218657 24510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:55.219682 24510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/data/instance:
uuid: "7a492a775a2a46859e2bd718cf4a243b"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "8ca803655458957901bcd20809c2a011"
server_key_iv: "1eb59641e5076aee8f3b4a1bbbe5dc7b"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.220008 24510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance:
uuid: "7a492a775a2a46859e2bd718cf4a243b"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "8ca803655458957901bcd20809c2a011"
server_key_iv: "1eb59641e5076aee8f3b4a1bbbe5dc7b"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.221226 24510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:12:55.221890 24524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.222047 24510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:55.222128 24510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/wal
uuid: "7a492a775a2a46859e2bd718cf4a243b"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "8ca803655458957901bcd20809c2a011"
server_key_iv: "1eb59641e5076aee8f3b4a1bbbe5dc7b"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.222193 24510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:55.230338 24510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:55.230643 24510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:55.230768 24510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:55.230962 24510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:55.231263 24510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:55.231305 24510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.231331 24510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:55.231355 24510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.236750 24510 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.65:45597
I20260927 07:12:55.236809 24637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.65:45597 every 8 connection(s)
I20260927 07:12:55.237099 24510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/data/info.pb
I20260927 07:12:55.240849 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 24510
I20260927 07:12:55.240928 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-0/wal/instance
I20260927 07:12:55.241108 4793 external_mini_cluster.cc:1428] Setting key a682294f7e72bf532b96f82223e88a3b
I20260927 07:12:55.242028 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.66:0
--local_ip_for_outbound_sockets=127.4.174.66
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40337
--builtin_ntp_servers=127.4.174.84:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:12:55.242203 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.237750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.65:40107 (local address 127.4.174.126:40337)
0927 07:12:55.237817 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.237819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.238039 (+ 220us) server_negotiation.cc:407] Connection header received
0927 07:12:55.238575 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.238577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.238592 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.238644 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:55.239299 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.239842 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.240793 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.241024 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.241083 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.241949 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:55.241950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:55.241951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:55.241956 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:55.241970 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:55.241974 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:55.242018 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.242091 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.242126 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 07:12:55.242414 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.237734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40337 (local address 127.4.174.65:40107)
0927 07:12:55.237921 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.237961 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:55.238459 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:55.238715 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:55.238719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:55.238731 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:55.239212 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.239216 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.239925 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.239926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.240706 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.240708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.241090 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.241091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.241238 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.241812 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:55.241813 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:55.241850 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:55.241877 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:55.242029 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:55.242034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:55.242036 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:12:55.242096 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:12:55.242196 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260927 07:12:55.242880 24638 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40337
I20260927 07:12:55.242995 24638 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:55.243281 24638 heartbeater.cc:511] Master 127.4.174.126:40337 requested a full tablet report, sending...
I20260927 07:12:55.243870 24450 ts_manager.cc:194] Registered new tserver with Master: 7a492a775a2a46859e2bd718cf4a243b (127.4.174.65:45597)
I20260927 07:12:55.244596 24450 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.65:40107
W20260927 07:12:55.321180 24641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:55.321393 24641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:55.321424 24641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:55.322613 24641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:55.322654 24641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:55.322685 24641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.66
I20260927 07:12:55.324054 24641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.174.66
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.66
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:55.324301 24641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:55.324622 24641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:55.329533 24649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:55.329555 24641 server_base.cc:1034] running on GCE node
W20260927 07:12:55.329505 24647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:55.329501 24646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:55.329918 24641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:55.330189 24641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:55.331321 24641 hybrid_clock.cc:648] HybridClock initialized: now 1790493175331309 us; error 29 us; skew 500 ppm
I20260927 07:12:55.332579 24641 webserver.cc:460] Webserver started at http://127.4.174.66:37833/ using document root <none> and password file <none>
I20260927 07:12:55.332816 24641 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:55.332865 24641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:55.332994 24641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:55.334033 24641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/data/instance:
uuid: "938fd71e086f4a52b8921bde771b6d2d"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "87fe033739a797873ab8dcc0e39055c8"
server_key_iv: "22f31b8665a0568464c262a83e4d8767"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.334355 24641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance:
uuid: "938fd71e086f4a52b8921bde771b6d2d"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "87fe033739a797873ab8dcc0e39055c8"
server_key_iv: "22f31b8665a0568464c262a83e4d8767"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.335640 24641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:12:55.336361 24655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.336511 24641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:55.336583 24641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/wal
uuid: "938fd71e086f4a52b8921bde771b6d2d"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "87fe033739a797873ab8dcc0e39055c8"
server_key_iv: "22f31b8665a0568464c262a83e4d8767"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.336649 24641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:55.370587 24641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:55.370913 24641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:55.371047 24641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:55.371272 24641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:55.371613 24641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:55.371649 24641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.371677 24641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:55.371694 24641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.376879 24641 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.66:39931
I20260927 07:12:55.376925 24768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.66:39931 every 8 connection(s)
I20260927 07:12:55.377250 24641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/data/info.pb
I20260927 07:12:55.377466 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 24641
I20260927 07:12:55.377578 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/wal/instance
I20260927 07:12:55.377779 4793 external_mini_cluster.cc:1428] Setting key add4291d138dbdad1092f6eac9ba7fe2
I20260927 07:12:55.378808 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.174.67:0
--local_ip_for_outbound_sockets=127.4.174.67
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40337
--builtin_ntp_servers=127.4.174.84:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:12:55.383050 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.377966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.66:49243 (local address 127.4.174.126:40337)
0927 07:12:55.378079 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.378081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.378300 (+ 219us) server_negotiation.cc:407] Connection header received
0927 07:12:55.378964 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.378966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.378978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.379004 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:55.379953 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.380655 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.381598 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.381811 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.381865 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.382766 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:55.382767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:55.382767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:55.382772 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:55.382784 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:55.382787 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:55.382825 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.382967 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.382991 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 07:12:55.383329 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.377989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40337 (local address 127.4.174.66:49243)
0927 07:12:55.378185 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.378220 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:55.378836 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:55.379188 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:55.379192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:55.379203 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:55.379861 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.379868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.380737 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.380738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.381527 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.381530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.381901 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.381903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.382066 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.382617 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:55.382617 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:55.382658 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:55.382686 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:55.382892 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:55.382899 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:55.382902 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:55.382984 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 07:12:55.383099 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260927 07:12:55.383816 24769 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40337
I20260927 07:12:55.383916 24769 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:55.384087 24769 heartbeater.cc:511] Master 127.4.174.126:40337 requested a full tablet report, sending...
I20260927 07:12:55.384465 24450 ts_manager.cc:194] Registered new tserver with Master: 938fd71e086f4a52b8921bde771b6d2d (127.4.174.66:39931)
I20260927 07:12:55.384950 24450 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.66:49243
W20260927 07:12:55.459887 24772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:55.460088 24772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:55.460108 24772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:55.461208 24772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:55.461242 24772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:55.461265 24772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.174.67
I20260927 07:12:55.462574 24772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.174.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.174.67
--webserver_port=0
--tserver_master_addrs=127.4.174.126:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.174.67
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:55.462759 24772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:55.462989 24772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:55.467774 24777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:55.467797 24778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:55.467790 24780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:55.467993 24772 server_base.cc:1034] running on GCE node
I20260927 07:12:55.468309 24772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:55.468547 24772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:55.469682 24772 hybrid_clock.cc:648] HybridClock initialized: now 1790493175469665 us; error 34 us; skew 500 ppm
I20260927 07:12:55.471050 24772 webserver.cc:460] Webserver started at http://127.4.174.67:40001/ using document root <none> and password file <none>
I20260927 07:12:55.471305 24772 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:55.471361 24772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:55.471496 24772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:55.472594 24772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/data/instance:
uuid: "bdf5d2dc0fdf4b73baddf587545e86d7"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "ca14454b7f754a6d879e4f4ffc27f832"
server_key_iv: "ad708c5b1afd7673037cf128bce99ba8"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.472942 24772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance:
uuid: "bdf5d2dc0fdf4b73baddf587545e86d7"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "ca14454b7f754a6d879e4f4ffc27f832"
server_key_iv: "ad708c5b1afd7673037cf128bce99ba8"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.474246 24772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:12:55.475069 24786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.475230 24772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.475315 24772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/wal
uuid: "bdf5d2dc0fdf4b73baddf587545e86d7"
format_stamp: "Formatted at 2026-09-27 07:12:55 on dist-test-slave-zw7p"
server_key: "ca14454b7f754a6d879e4f4ffc27f832"
server_key_iv: "ad708c5b1afd7673037cf128bce99ba8"
server_key_version: "encryptionkey@0"
I20260927 07:12:55.475385 24772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:55.487231 24772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:55.487512 24772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:55.487630 24772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:55.487833 24772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:12:55.488147 24772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:12:55.488181 24772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.488210 24772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:12:55.488234 24772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:55.493559 24772 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.67:33131
I20260927 07:12:55.493613 24899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.67:33131 every 8 connection(s)
I20260927 07:12:55.493928 24772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/data/info.pb
I20260927 07:12:55.494498 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 24772
I20260927 07:12:55.494599 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/wal/instance
I20260927 07:12:55.494792 4793 external_mini_cluster.cc:1428] Setting key e03e6f61555f6047adb46565d60dd218
I20260927 07:12:55.499238 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.494921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:52701 (local address 127.4.174.126:40337)
0927 07:12:55.494968 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.494970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.495297 (+ 327us) server_negotiation.cc:407] Connection header received
0927 07:12:55.495891 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.495892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.495901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.495946 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:55.496577 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.497069 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.498047 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.498224 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.498263 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.499026 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:55.499027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:55.499027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:55.499032 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:55.499044 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:55.499047 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:55.499083 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.499155 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.499178 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:12:55.499447 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.494928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:40337 (local address 127.4.174.67:52701)
0927 07:12:55.495166 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.495211 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:55.495815 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:55.496017 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:55.496020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:55.496028 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:12:55.496518 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.496522 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.497160 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.497162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.497979 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.497981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.498274 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.498275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.498382 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.498913 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:12:55.498913 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:12:55.498943 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:12:55.498967 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:12:55.499085 (+ 118us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:12:55.499087 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:12:55.499089 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 07:12:55.499153 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 07:12:55.499235 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":78,"threads_started":1}
I20260927 07:12:55.499883 24900 heartbeater.cc:347] Connected to a master server at 127.4.174.126:40337
I20260927 07:12:55.499965 24900 heartbeater.cc:464] Registering TS with master...
I20260927 07:12:55.500136 24900 heartbeater.cc:511] Master 127.4.174.126:40337 requested a full tablet report, sending...
I20260927 07:12:55.500452 24450 ts_manager.cc:194] Registered new tserver with Master: bdf5d2dc0fdf4b73baddf587545e86d7 (127.4.174.67:33131)
I20260927 07:12:55.500820 24450 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.174.67:52701
I20260927 07:12:55.506649 4793 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:12:55.510653 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.507961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.4.174.126:40337)
0927 07:12:55.508039 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.508041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.508118 (+ 77us) server_negotiation.cc:407] Connection header received
0927 07:12:55.508164 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.508164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.508175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.508224 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:55.508355 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.508833 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.509560 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.509753 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.509812 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.510400 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:55.510402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:55.510402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:55.510407 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:55.510423 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:55.510427 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:55.510475 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.510512 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.510539 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 07:12:55.512066 24450 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:12:55.513603 24450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:12:55.519548 24911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.515885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48224 (local address 127.4.174.67:33131)
0927 07:12:55.516158 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.516161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.516170 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:55.516207 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.516208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.516221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.516295 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:55.516480 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.517222 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.518502 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.519176 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.519236 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.519328 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.519427 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.519463 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":75,"threads_started":1}
I20260927 07:12:55.519548 24910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.515712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.67:33131 (local address 127.0.0.1:48224)
0927 07:12:55.515897 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.515935 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:55.516018 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:55.516315 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:55.516318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:55.516321 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:55.516387 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.516389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.517300 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.517301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.518407 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.518409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.519249 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.519250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.519324 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.519338 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 07:12:55.519403 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 07:12:55.519454 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
I20260927 07:12:55.519774 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.515712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45597 (local address 127.0.0.1:40512)
0927 07:12:55.516125 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.516133 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:55.516174 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:55.516441 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:55.516442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:55.516444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:55.516486 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.516488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.517769 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.517770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.518705 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.518707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.519565 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.519566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.519628 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.519632 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:12:55.519660 (+ 28us) client_negotiation.cc:240] Negotiation successful
0927 07:12:55.519698 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":48,"spinlock_wait_cycles":384,"thread_start_us":96,"threads_started":1}
I20260927 07:12:55.520049 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.515722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39931 (local address 127.0.0.1:57974)
0927 07:12:55.516240 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.516247 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:55.516286 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:55.516528 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:55.516529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:55.516531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:55.516559 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.516560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.517765 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.517766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.518846 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.518848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.519778 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.519779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.519903 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.519908 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:12:55.519951 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 07:12:55.519987 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":94,"threads_started":1}
I20260927 07:12:55.520119 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.515904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.4.174.65:45597)
0927 07:12:55.516105 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.516109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.516169 (+ 60us) server_negotiation.cc:407] Connection header received
0927 07:12:55.516233 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.516235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.516250 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.516358 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:55.516554 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.517671 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.518772 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.519467 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.519886 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.519981 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.520005 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.520041 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":98,"threads_started":1}
I20260927 07:12:55.520541 24834 tablet_service.cc:1467] Processing CreateTablet for tablet ef808521ae704764b26eb7c7fb65b9f6 (DEFAULT_TABLE table=test-table [id=3c013fa95e134afc8a08e7848ba83929]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:55.520670 24572 tablet_service.cc:1467] Processing CreateTablet for tablet ef808521ae704764b26eb7c7fb65b9f6 (DEFAULT_TABLE table=test-table [id=3c013fa95e134afc8a08e7848ba83929]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:55.520913 24834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef808521ae704764b26eb7c7fb65b9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:55.521056 24572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef808521ae704764b26eb7c7fb65b9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:55.522781 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.515953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.4.174.66:39931)
0927 07:12:55.516112 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.516117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.516342 (+ 225us) server_negotiation.cc:407] Connection header received
0927 07:12:55.516387 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.516389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.516403 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.516464 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:55.516626 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.517691 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.518922 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.519599 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.519654 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.519736 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.522650 (+ 2914us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.522701 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260927 07:12:55.523471 24703 tablet_service.cc:1467] Processing CreateTablet for tablet ef808521ae704764b26eb7c7fb65b9f6 (DEFAULT_TABLE table=test-table [id=3c013fa95e134afc8a08e7848ba83929]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:12:55.523806 24703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef808521ae704764b26eb7c7fb65b9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:55.523838 24914 tablet_bootstrap.cc:492] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Bootstrap starting.
I20260927 07:12:55.524333 24914 tablet_bootstrap.cc:654] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:55.524561 24914 log.cc:824] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:55.524814 24915 tablet_bootstrap.cc:492] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: Bootstrap starting.
I20260927 07:12:55.525317 24914 tablet_bootstrap.cc:492] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: No bootstrap required, opened a new log
I20260927 07:12:55.525373 24914 ts_tablet_manager.cc:1397] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:55.525393 24915 tablet_bootstrap.cc:654] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:55.525684 24915 log.cc:824] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:55.526405 24918 tablet_bootstrap.cc:492] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Bootstrap starting.
I20260927 07:12:55.526460 24915 tablet_bootstrap.cc:492] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: No bootstrap required, opened a new log
I20260927 07:12:55.526530 24915 ts_tablet_manager.cc:1397] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:55.527040 24918 tablet_bootstrap.cc:654] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:55.527343 24918 log.cc:824] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:55.527917 24918 tablet_bootstrap.cc:492] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: No bootstrap required, opened a new log
I20260927 07:12:55.527971 24918 ts_tablet_manager.cc:1397] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:12:55.527894 24914 raft_consensus.cc:348] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.528074 24914 raft_consensus.cc:374] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:55.528118 24914 raft_consensus.cc:729] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf5d2dc0fdf4b73baddf587545e86d7, State: Initialized, Role: FOLLOWER
I20260927 07:12:55.528237 24914 consensus_queue.cc:260] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.528486 24914 ts_tablet_manager.cc:1428] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:12:55.528623 24900 heartbeater.cc:503] Master 127.4.174.126:40337 was elected leader, sending a full tablet report...
I20260927 07:12:55.528828 24915 raft_consensus.cc:348] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.528972 24915 raft_consensus.cc:374] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:55.529006 24915 raft_consensus.cc:729] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a492a775a2a46859e2bd718cf4a243b, State: Initialized, Role: FOLLOWER
I20260927 07:12:55.529088 24915 consensus_queue.cc:260] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.529325 24638 heartbeater.cc:503] Master 127.4.174.126:40337 was elected leader, sending a full tablet report...
I20260927 07:12:55.529524 24915 ts_tablet_manager.cc:1428] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 07:12:55.530221 24918 raft_consensus.cc:348] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.530345 24918 raft_consensus.cc:374] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:55.530376 24918 raft_consensus.cc:729] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938fd71e086f4a52b8921bde771b6d2d, State: Initialized, Role: FOLLOWER
I20260927 07:12:55.530480 24918 consensus_queue.cc:260] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.530709 24918 ts_tablet_manager.cc:1428] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260927 07:12:55.530768 24769 heartbeater.cc:503] Master 127.4.174.126:40337 was elected leader, sending a full tablet report...
W20260927 07:12:55.627816 24770 tablet.cc:2369] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:12:55.715130 24920 raft_consensus.cc:482] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:12:55.715256 24920 raft_consensus.cc:504] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.715619 24920 leader_election.cc:288] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a492a775a2a46859e2bd718cf4a243b (127.4.174.65:45597), 938fd71e086f4a52b8921bde771b6d2d (127.4.174.66:39931)
I20260927 07:12:55.718295 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.715876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:58627 (local address 127.4.174.65:45597)
0927 07:12:55.715918 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.715920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.716075 (+ 155us) server_negotiation.cc:407] Connection header received
0927 07:12:55.716109 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.716110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.716123 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.716184 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:55.716317 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.716763 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.717635 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.717998 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.718061 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.718069 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.718196 (+ 127us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.718232 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 07:12:55.718297 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.715849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.174.67:54043 (local address 127.4.174.66:39931)
0927 07:12:55.715890 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.715892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.715902 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:12:55.715958 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.715959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.715967 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.716036 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:12:55.716192 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.716718 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.717628 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.717981 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.718032 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.718040 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.718186 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.718220 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:12:55.718295 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.715788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.66:39931 (local address 127.4.174.67:54043)
0927 07:12:55.715837 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.715848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:55.715892 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:55.716051 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:55.716052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:55.716055 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:55.716129 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.716131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.716792 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.716793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.717540 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.717543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.718059 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.718060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.718123 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.718126 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:55.718167 (+ 41us) client_negotiation.cc:240] Negotiation successful
0927 07:12:55.718208 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 07:12:55.718308 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.715790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45597 (local address 127.4.174.67:58627)
0927 07:12:55.716013 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.716021 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:12:55.716066 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:12:55.716192 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:12:55.716192 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:12:55.716194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:12:55.716257 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.716259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.716833 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.716834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.717551 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:12:55.717553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.718070 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.718071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:12:55.718132 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.718135 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 07:12:55.718181 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 07:12:55.718216 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":49,"thread_start_us":91,"threads_started":1}
I20260927 07:12:55.718650 24723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef808521ae704764b26eb7c7fb65b9f6" candidate_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938fd71e086f4a52b8921bde771b6d2d" is_pre_election: true
I20260927 07:12:55.718650 24592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef808521ae704764b26eb7c7fb65b9f6" candidate_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a492a775a2a46859e2bd718cf4a243b" is_pre_election: true
I20260927 07:12:55.718787 24592 raft_consensus.cc:2454] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf5d2dc0fdf4b73baddf587545e86d7 in term 0.
I20260927 07:12:55.718796 24723 raft_consensus.cc:2454] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf5d2dc0fdf4b73baddf587545e86d7 in term 0.
I20260927 07:12:55.718981 24788 leader_election.cc:302] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a492a775a2a46859e2bd718cf4a243b, bdf5d2dc0fdf4b73baddf587545e86d7; no voters:
I20260927 07:12:55.719115 24920 raft_consensus.cc:2781] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:12:55.719174 24920 raft_consensus.cc:482] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:12:55.719190 24920 raft_consensus.cc:3037] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:55.719839 24920 raft_consensus.cc:504] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.719954 24920 leader_election.cc:288] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [CANDIDATE]: Term 1 election: Requested vote from peers 7a492a775a2a46859e2bd718cf4a243b (127.4.174.65:45597), 938fd71e086f4a52b8921bde771b6d2d (127.4.174.66:39931)
I20260927 07:12:55.720111 24723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef808521ae704764b26eb7c7fb65b9f6" candidate_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938fd71e086f4a52b8921bde771b6d2d"
I20260927 07:12:55.720111 24592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef808521ae704764b26eb7c7fb65b9f6" candidate_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a492a775a2a46859e2bd718cf4a243b"
I20260927 07:12:55.720186 24592 raft_consensus.cc:3037] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:55.720186 24723 raft_consensus.cc:3037] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:55.721016 24592 raft_consensus.cc:2454] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf5d2dc0fdf4b73baddf587545e86d7 in term 1.
I20260927 07:12:55.721016 24723 raft_consensus.cc:2454] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf5d2dc0fdf4b73baddf587545e86d7 in term 1.
I20260927 07:12:55.721222 24788 leader_election.cc:302] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a492a775a2a46859e2bd718cf4a243b, bdf5d2dc0fdf4b73baddf587545e86d7; no voters:
I20260927 07:12:55.721376 24920 raft_consensus.cc:2781] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:55.721519 24920 raft_consensus.cc:686] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 1 LEADER]: Becoming Leader. State: Replica: bdf5d2dc0fdf4b73baddf587545e86d7, State: Running, Role: LEADER
I20260927 07:12:55.721603 24920 consensus_queue.cc:237] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } }
I20260927 07:12:55.722438 24449 catalog_manager.cc:5495] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf5d2dc0fdf4b73baddf587545e86d7 (127.4.174.67). New cstate: current_term: 1 leader_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdf5d2dc0fdf4b73baddf587545e86d7" member_type: VOTER last_known_addr { host: "127.4.174.67" port: 33131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 } health_report { overall_health: UNKNOWN } } }
W20260927 07:12:55.737766 24639 tablet.cc:2369] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:12:55.744431 24901 tablet.cc:2369] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:12:55.744941 24911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.742651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.4.174.67:33131)
0927 07:12:55.742818 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:55.742820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:55.742830 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:12:55.742874 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:55.742875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:55.742898 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:55.742964 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:12:55.743107 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.743580 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.744272 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:55.744462 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:55.744515 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:55.744672 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:12:55.744771 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:12:55.744839 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:55.744853 (+ 14us) server_negotiation.cc:299] Negotiation successful
0927 07:12:55.744871 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260927 07:12:55.746570 24723 raft_consensus.cc:1261] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 1 FOLLOWER]: Refusing update from remote peer bdf5d2dc0fdf4b73baddf587545e86d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:55.746628 24592 raft_consensus.cc:1261] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 1 FOLLOWER]: Refusing update from remote peer bdf5d2dc0fdf4b73baddf587545e86d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:12:55.746948 24924 consensus_queue.cc:1035] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "938fd71e086f4a52b8921bde771b6d2d" member_type: VOTER last_known_addr { host: "127.4.174.66" port: 39931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:55.747097 24920 consensus_queue.cc:1035] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a492a775a2a46859e2bd718cf4a243b" member_type: VOTER last_known_addr { host: "127.4.174.65" port: 45597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:12:55.749091 24928 mvcc.cc:205] Tried to move back new op lower bound from 7333860047855546368 to 7333860047756263424. Current Snapshot: MvccSnapshot[applied={T|T < 7333860047855546368}]
I20260927 07:12:55.749878 24929 mvcc.cc:205] Tried to move back new op lower bound from 7333860047855546368 to 7333860047756263424. Current Snapshot: MvccSnapshot[applied={T|T < 7333860047855546368}]
I20260927 07:12:55.751868 24449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:55.751948 24449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:12:55.752655 24449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5cef726e8a4d432d9cb17df3f8207870: Sending DeleteTablet for 3 replicas of tablet ef808521ae704764b26eb7c7fb65b9f6
I20260927 07:12:55.752995 24834 tablet_service.cc:1514] Processing DeleteTablet for tablet ef808521ae704764b26eb7c7fb65b9f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:55 UTC) from {username='slave'} at 127.0.0.1:48224
I20260927 07:12:55.753063 24572 tablet_service.cc:1514] Processing DeleteTablet for tablet ef808521ae704764b26eb7c7fb65b9f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:55 UTC) from {username='slave'} at 127.0.0.1:40512
I20260927 07:12:55.753196 24703 tablet_service.cc:1514] Processing DeleteTablet for tablet ef808521ae704764b26eb7c7fb65b9f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:12:55 UTC) from {username='slave'} at 127.0.0.1:57974
I20260927 07:12:55.753321 24938 tablet_replica.cc:330] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: stopping tablet replica
I20260927 07:12:55.753449 24938 raft_consensus.cc:2229] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:55.753506 24937 tablet_replica.cc:330] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: stopping tablet replica
I20260927 07:12:55.753590 24937 raft_consensus.cc:2229] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:12:55.753700 24937 raft_consensus.cc:2258] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:55.753832 24939 tablet_replica.cc:330] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: stopping tablet replica
I20260927 07:12:55.753924 24939 raft_consensus.cc:2229] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:12:55.753922 24938 raft_consensus.cc:2258] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:55.754020 24939 raft_consensus.cc:2258] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:12:55.754444 24938 ts_tablet_manager.cc:1905] T ef808521ae704764b26eb7c7fb65b9f6 P 7a492a775a2a46859e2bd718cf4a243b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:55.754444 24937 ts_tablet_manager.cc:1905] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:55.754601 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 24510
I20260927 07:12:55.756457 24939 ts_tablet_manager.cc:1905] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:12:55.757615 24937 ts_tablet_manager.cc:1918] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:55.757690 24937 log.cc:1196] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-2/wal/wals/ef808521ae704764b26eb7c7fb65b9f6
I20260927 07:12:55.757917 24937 ts_tablet_manager.cc:1939] T ef808521ae704764b26eb7c7fb65b9f6 P bdf5d2dc0fdf4b73baddf587545e86d7: Deleting consensus metadata
I20260927 07:12:55.758338 24437 catalog_manager.cc:4841] TS bdf5d2dc0fdf4b73baddf587545e86d7 (127.4.174.67:33131): tablet ef808521ae704764b26eb7c7fb65b9f6 (table test-table [id=3c013fa95e134afc8a08e7848ba83929]) successfully deleted
I20260927 07:12:55.759509 24939 ts_tablet_manager.cc:1918] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:12:55.759585 24939 log.cc:1196] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Deleting WAL directory at /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492940996973-4793-0/minicluster-data/ts-1/wal/wals/ef808521ae704764b26eb7c7fb65b9f6
I20260927 07:12:55.759801 24939 ts_tablet_manager.cc:1939] T ef808521ae704764b26eb7c7fb65b9f6 P 938fd71e086f4a52b8921bde771b6d2d: Deleting consensus metadata
I20260927 07:12:55.760216 24436 catalog_manager.cc:4841] TS 938fd71e086f4a52b8921bde771b6d2d (127.4.174.66:39931): tablet ef808521ae704764b26eb7c7fb65b9f6 (table test-table [id=3c013fa95e134afc8a08e7848ba83929]) successfully deleted
W20260927 07:12:55.761430 24435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.174.65:45597 (error 108)
I20260927 07:12:55.761901 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:55.761768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.65:45597 (local address 127.0.0.1:40514)
0927 07:12:55.761821 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0927 07:12:55.761865 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.65:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 07:12:55.761909 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 24641
W20260927 07:12:55.763352 24435 catalog_manager.cc:4584] TS 7a492a775a2a46859e2bd718cf4a243b (127.4.174.65:45597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef808521ae704764b26eb7c7fb65b9f6: Network error: Client connection negotiation failed: client connection to 127.4.174.65:45597: connect: Connection refused (error 111)
I20260927 07:12:55.766702 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 24772
I20260927 07:12:55.771198 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 24419
2026-09-27T07:12:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (846 ms)
[----------] 6 tests from EncryptionPolicyTest (8069 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:12:55 dist-test-slave-zw7p krb5kdc[24943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:12:55 dist-test-slave-zw7p krb5kdc[24943](info): set up 2 sockets
Sep 27 07:12:55 dist-test-slave-zw7p krb5kdc[24943](info): commencing operation
krb5kdc: starting...
W20260927 07:12:56.718264 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:12:56 dist-test-slave-zw7p krb5kdc[24943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:12:56.828608 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:56.906164 24956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:56.906373 24956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:56.906404 24956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:56.907574 24956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:56.907612 24956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:56.907629 24956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:56.907645 24956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:56.907660 24956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:56.909072 24956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:56.909318 24956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:56.909519 24956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:56.914005 24962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:56.914004 24964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:56.914191 24956 server_base.cc:1034] running on GCE node
W20260927 07:12:56.914299 24961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:56.914544 24956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:56.914849 24956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:56.916003 24956 hybrid_clock.cc:648] HybridClock initialized: now 1790493176915991 us; error 28 us; skew 500 ppm
Sep 27 07:12:56 dist-test-slave-zw7p krb5kdc[24943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:56.918702 24956 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:12:56.919119 24956 webserver.cc:460] Webserver started at http://127.0.0.1:34557/ using document root <none> and password file <none>
I20260927 07:12:56.919324 24956 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:56.919373 24956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:56.919514 24956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:56.920639 24956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "16e3beb038fb4b88bae50688e79630ff"
format_stamp: "Formatted at 2026-09-27 07:12:56 on dist-test-slave-zw7p"
server_key: "502db0dc0fbba9f64af9fcf59986a133"
server_key_iv: "d395571f2cebc812ffb842518c3d9ecd"
server_key_version: "encryptionkey@0"
I20260927 07:12:56.921028 24956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "16e3beb038fb4b88bae50688e79630ff"
format_stamp: "Formatted at 2026-09-27 07:12:56 on dist-test-slave-zw7p"
server_key: "502db0dc0fbba9f64af9fcf59986a133"
server_key_iv: "d395571f2cebc812ffb842518c3d9ecd"
server_key_version: "encryptionkey@0"
I20260927 07:12:56.922677 24956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:12:56.923524 24971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:56.923683 24956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:12:56.923776 24956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "16e3beb038fb4b88bae50688e79630ff"
format_stamp: "Formatted at 2026-09-27 07:12:56 on dist-test-slave-zw7p"
server_key: "502db0dc0fbba9f64af9fcf59986a133"
server_key_iv: "d395571f2cebc812ffb842518c3d9ecd"
server_key_version: "encryptionkey@0"
I20260927 07:12:56.923832 24956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:56.939095 24956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:56.940987 24956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:56.941094 24956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:56.944744 24956 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39747
I20260927 07:12:56.944787 25023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39747 every 8 connection(s)
I20260927 07:12:56.945114 24956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:56.946069 25024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:56.948398 25024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff: Bootstrap starting.
I20260927 07:12:56.948945 25024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:56.949200 25024 log.cc:824] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:56.949924 25024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff: No bootstrap required, opened a new log
I20260927 07:12:56.951566 25024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e3beb038fb4b88bae50688e79630ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260927 07:12:56.951707 25024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:56.951740 25024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e3beb038fb4b88bae50688e79630ff, State: Initialized, Role: FOLLOWER
I20260927 07:12:56.951867 25024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e3beb038fb4b88bae50688e79630ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260927 07:12:56.951953 25024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:56.951987 25024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:56.952033 25024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:56.952531 25024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e3beb038fb4b88bae50688e79630ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260927 07:12:56.952621 25024 leader_election.cc:302] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16e3beb038fb4b88bae50688e79630ff; no voters:
I20260927 07:12:56.952782 25024 leader_election.cc:288] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:56.952821 25027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:56.953003 25027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [term 1 LEADER]: Becoming Leader. State: Replica: 16e3beb038fb4b88bae50688e79630ff, State: Running, Role: LEADER
I20260927 07:12:56.953001 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 24956
I20260927 07:12:56.953047 25024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:56.953090 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:56.953099 25027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e3beb038fb4b88bae50688e79630ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260927 07:12:56.953271 4793 external_mini_cluster.cc:1428] Setting key 7a079af6259183dc60d3d6dfb3ac8b19
I20260927 07:12:56.953465 25029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16e3beb038fb4b88bae50688e79630ff. Latest consensus state: current_term: 1 leader_uuid: "16e3beb038fb4b88bae50688e79630ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e3beb038fb4b88bae50688e79630ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } } }
I20260927 07:12:56.953542 25029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:56.953593 25028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16e3beb038fb4b88bae50688e79630ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e3beb038fb4b88bae50688e79630ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } } }
I20260927 07:12:56.953668 25028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:56.953745 25034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:56.954478 25034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:56.956494 25034 catalog_manager.cc:1347] Generated new cluster ID: e28730131475434fa0d8d42b81ecf95a
I20260927 07:12:56.956543 25034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:12:56 dist-test-slave-zw7p krb5kdc[24943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 07:12:56.964576 25039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:56.953921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60265 (local address 127.0.0.1:39747)
0927 07:12:56.954156 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:56.954166 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:56.954198 (+ 32us) server_negotiation.cc:407] Connection header received
0927 07:12:56.954322 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:56.954329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:56.954394 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:56.954549 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:56.955453 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:56.956721 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:56.957790 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:56.958380 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:56.958469 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:56.961122 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:56.961127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:56.961127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:56.961139 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:56.963013 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:56.963407 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:56.963410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:56.963410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:56.963526 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:56.963728 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:56.963729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:56.963729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:56.964029 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:56.964197 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:56.964274 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 07:12:56.964369 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":95,"threads_started":1}
I20260927 07:12:56.965706 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:12:56.977448 25034 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:56.977792 25039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:56.970813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38217 (local address 127.0.0.1:39747)
0927 07:12:56.970849 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:56.970851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:56.970978 (+ 127us) server_negotiation.cc:407] Connection header received
0927 07:12:56.971034 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:56.971035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:56.971045 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:56.971092 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:56.971662 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:56.972382 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:56.973372 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:56.973654 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:56.973705 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:56.975024 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:56.975025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:56.975025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:12:56.975032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:56.976495 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:56.976896 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:56.976897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:56.976897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:56.976963 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:12:56.977196 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:12:56.977197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:12:56.977197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:12:56.977322 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:56.977418 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:56.977608 (+ 190us) server_negotiation.cc:299] Negotiation successful
0927 07:12:56.977669 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:12:56.978048 25034 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:56.979768 25034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16e3beb038fb4b88bae50688e79630ff: Generated new TSK 0
I20260927 07:12:56.979918 25034 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:12:56.982156 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 24956
W20260927 07:12:56.985881 24941 connection.cc:535] client connection to 127.0.0.1:39747 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39747): Connection reset by peer (error 104)
2026-09-27T07:12:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1209 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:12:57 dist-test-slave-zw7p krb5kdc[25055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:12:57 dist-test-slave-zw7p krb5kdc[25055](info): set up 2 sockets
Sep 27 07:12:57 dist-test-slave-zw7p krb5kdc[25055](info): commencing operation
krb5kdc: starting...
W20260927 07:12:57.934275 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:12:58 dist-test-slave-zw7p krb5kdc[25055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:12:58.046640 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46603
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:58.125523 25068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:58.125734 25068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:58.125765 25068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:58.126945 25068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:58.126986 25068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:58.127003 25068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:58.127019 25068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:58.127033 25068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:58.128494 25068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46603
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:58.128736 25068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:58.128939 25068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:58.133498 25074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:58.133494 25073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:58.133539 25076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:58.133679 25068 server_base.cc:1034] running on GCE node
I20260927 07:12:58.133919 25068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:58.134199 25068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:58.135380 25068 hybrid_clock.cc:648] HybridClock initialized: now 1790493178135360 us; error 37 us; skew 500 ppm
Sep 27 07:12:58 dist-test-slave-zw7p krb5kdc[25055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493178, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:58.137748 25068 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:12:58.138171 25068 webserver.cc:460] Webserver started at http://127.0.0.1:34007/ using document root <none> and password file <none>
I20260927 07:12:58.138387 25068 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:58.138437 25068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:58.138579 25068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:58.139604 25068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "df9d74fc54c34934bdd901e97955bcd5"
format_stamp: "Formatted at 2026-09-27 07:12:58 on dist-test-slave-zw7p"
server_key: "384ef81c8c7bdcda7d3289a2ea16bb17"
server_key_iv: "3bde46af6b141cdb2d6cd8a6dee89eb9"
server_key_version: "encryptionkey@0"
I20260927 07:12:58.139918 25068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "df9d74fc54c34934bdd901e97955bcd5"
format_stamp: "Formatted at 2026-09-27 07:12:58 on dist-test-slave-zw7p"
server_key: "384ef81c8c7bdcda7d3289a2ea16bb17"
server_key_iv: "3bde46af6b141cdb2d6cd8a6dee89eb9"
server_key_version: "encryptionkey@0"
I20260927 07:12:58.141113 25068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:12:58.141793 25083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:58.141965 25068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:58.142045 25068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "df9d74fc54c34934bdd901e97955bcd5"
format_stamp: "Formatted at 2026-09-27 07:12:58 on dist-test-slave-zw7p"
server_key: "384ef81c8c7bdcda7d3289a2ea16bb17"
server_key_iv: "3bde46af6b141cdb2d6cd8a6dee89eb9"
server_key_version: "encryptionkey@0"
I20260927 07:12:58.142107 25068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:58.159809 25068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:58.161679 25068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:58.161810 25068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:58.165688 25068 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46603
I20260927 07:12:58.165731 25135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46603 every 8 connection(s)
I20260927 07:12:58.166061 25068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:58.167084 25136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:58.169449 25136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5: Bootstrap starting.
I20260927 07:12:58.170053 25136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:58.170336 25136 log.cc:824] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:58.171162 25136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5: No bootstrap required, opened a new log
I20260927 07:12:58.171336 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25068
I20260927 07:12:58.171432 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:58.171622 4793 external_mini_cluster.cc:1428] Setting key 1264d236a651f6f05718a388c03c913d
I20260927 07:12:58.173328 25136 raft_consensus.cc:348] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9d74fc54c34934bdd901e97955bcd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46603 } }
I20260927 07:12:58.173478 25136 raft_consensus.cc:374] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:58.173511 25136 raft_consensus.cc:729] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9d74fc54c34934bdd901e97955bcd5, State: Initialized, Role: FOLLOWER
I20260927 07:12:58.173640 25136 consensus_queue.cc:260] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9d74fc54c34934bdd901e97955bcd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46603 } }
I20260927 07:12:58.173718 25136 raft_consensus.cc:388] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:58.173749 25136 raft_consensus.cc:482] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:58.173799 25136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:58.174388 25136 raft_consensus.cc:504] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9d74fc54c34934bdd901e97955bcd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46603 } }
I20260927 07:12:58.174486 25136 leader_election.cc:302] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df9d74fc54c34934bdd901e97955bcd5; no voters:
I20260927 07:12:58.174695 25136 leader_election.cc:288] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:58.174719 25141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:58.174856 25141 raft_consensus.cc:686] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [term 1 LEADER]: Becoming Leader. State: Replica: df9d74fc54c34934bdd901e97955bcd5, State: Running, Role: LEADER
I20260927 07:12:58.174970 25136 sys_catalog.cc:564] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:58.175004 25141 consensus_queue.cc:237] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9d74fc54c34934bdd901e97955bcd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46603 } }
I20260927 07:12:58.175369 25143 sys_catalog.cc:455] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df9d74fc54c34934bdd901e97955bcd5. Latest consensus state: current_term: 1 leader_uuid: "df9d74fc54c34934bdd901e97955bcd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9d74fc54c34934bdd901e97955bcd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46603 } } }
I20260927 07:12:58.175472 25143 sys_catalog.cc:458] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:58.175366 25142 sys_catalog.cc:455] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df9d74fc54c34934bdd901e97955bcd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9d74fc54c34934bdd901e97955bcd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46603 } } }
I20260927 07:12:58.175540 25142 sys_catalog.cc:458] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:58.175833 25147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:58.176358 25147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:58.177863 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:58.172122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34849 (local address 127.0.0.1:46603)
0927 07:12:58.172287 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:58.172296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:58.172322 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:12:58.172880 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:58.172887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:58.172938 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:58.173054 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:58.173286 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:58.174468 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:58.175539 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:58.176171 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:58.176298 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:58.177213 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:58.177217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:58.177217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:58.177410 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:58.177449 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:58.177464 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:58.177547 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:58.177677 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 07:12:58.177704 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260927 07:12:58.178814 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:12:58.183274 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:58.180155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43893 (local address 127.0.0.1:46603)
0927 07:12:58.180213 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:58.180215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:58.180224 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:12:58.180263 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:58.180264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:58.180274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:58.180325 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:58.180476 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:58.180945 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:58.181982 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:12:58.182230 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:12:58.182287 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:12:58.183037 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:58.183038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:58.183039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:58.183043 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:58.183057 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:58.183061 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:58.183117 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:58.183206 (+ 89us) server_negotiation.cc:299] Negotiation successful
0927 07:12:58.183217 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 07:12:58.183948 25147 catalog_manager.cc:1347] Generated new cluster ID: 6d64de12db85499cabffa3f983ba58b6
I20260927 07:12:58.184012 25147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:58.194386 25147 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:58.194952 25147 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:58.202723 25147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df9d74fc54c34934bdd901e97955bcd5: Generated new TSK 0
I20260927 07:12:58.202945 25147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:12:58.213302 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25068
W20260927 07:12:58.217237 25053 connection.cc:535] client connection to 127.0.0.1:46603 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46603): Connection reset by peer (error 104)
2026-09-27T07:12:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1231 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:12:58 dist-test-slave-zw7p krb5kdc[25167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:12:58 dist-test-slave-zw7p krb5kdc[25167](info): set up 2 sockets
Sep 27 07:12:58 dist-test-slave-zw7p krb5kdc[25167](info): commencing operation
krb5kdc: starting...
W20260927 07:12:59.162636 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:12:59 dist-test-slave-zw7p krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:12:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:12:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:12:59.273285 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:12:59.353197 25180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:12:59.353407 25180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:12:59.353434 25180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:12:59.354609 25180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:12:59.354650 25180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:12:59.354663 25180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:12:59.354674 25180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:12:59.354684 25180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:12:59.356142 25180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:12:59.356361 25180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:12:59.356573 25180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:12:59.361073 25188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:12:59.361092 25185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:59.361160 25180 server_base.cc:1034] running on GCE node
W20260927 07:12:59.361091 25186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:12:59.361515 25180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:12:59.361796 25180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:12:59.362984 25180 hybrid_clock.cc:648] HybridClock initialized: now 1790493179362973 us; error 30 us; skew 500 ppm
Sep 27 07:12:59 dist-test-slave-zw7p krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:12:59.365283 25180 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:12:59.365711 25180 webserver.cc:460] Webserver started at http://127.0.0.1:46371/ using document root <none> and password file <none>
I20260927 07:12:59.365927 25180 fs_manager.cc:359] Metadata directory not provided
I20260927 07:12:59.365974 25180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:12:59.366089 25180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:12:59.367153 25180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "5d6a2003b3634857a51aa1766bcc791c"
format_stamp: "Formatted at 2026-09-27 07:12:59 on dist-test-slave-zw7p"
server_key: "b16e63cefa2e1deafd5b99d77be21734"
server_key_iv: "9e781fa00c1a2a2828a7294cd4612b5e"
server_key_version: "encryptionkey@0"
I20260927 07:12:59.367483 25180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "5d6a2003b3634857a51aa1766bcc791c"
format_stamp: "Formatted at 2026-09-27 07:12:59 on dist-test-slave-zw7p"
server_key: "b16e63cefa2e1deafd5b99d77be21734"
server_key_iv: "9e781fa00c1a2a2828a7294cd4612b5e"
server_key_version: "encryptionkey@0"
I20260927 07:12:59.368635 25180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:12:59.369298 25195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:12:59.369503 25180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:12:59.369596 25180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "5d6a2003b3634857a51aa1766bcc791c"
format_stamp: "Formatted at 2026-09-27 07:12:59 on dist-test-slave-zw7p"
server_key: "b16e63cefa2e1deafd5b99d77be21734"
server_key_iv: "9e781fa00c1a2a2828a7294cd4612b5e"
server_key_version: "encryptionkey@0"
I20260927 07:12:59.369655 25180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:12:59.372910 25180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:12:59.374724 25180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:12:59.374852 25180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:12:59.378538 25180 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40635
I20260927 07:12:59.378609 25247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40635 every 8 connection(s)
I20260927 07:12:59.378903 25180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:12:59.379791 25248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:12:59.381911 25248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c: Bootstrap starting.
I20260927 07:12:59.382441 25248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c: Neither blocks nor log segments found. Creating new log.
I20260927 07:12:59.382701 25248 log.cc:824] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:12:59.383311 25248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c: No bootstrap required, opened a new log
I20260927 07:12:59.384824 25248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d6a2003b3634857a51aa1766bcc791c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40635 } }
I20260927 07:12:59.384958 25248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:12:59.384977 25248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d6a2003b3634857a51aa1766bcc791c, State: Initialized, Role: FOLLOWER
I20260927 07:12:59.385095 25248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d6a2003b3634857a51aa1766bcc791c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40635 } }
I20260927 07:12:59.385154 25248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:12:59.385177 25248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:12:59.385226 25248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:12:59.385707 25248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d6a2003b3634857a51aa1766bcc791c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40635 } }
I20260927 07:12:59.385793 25248 leader_election.cc:302] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d6a2003b3634857a51aa1766bcc791c; no voters:
I20260927 07:12:59.385936 25248 leader_election.cc:288] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:12:59.386006 25251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:12:59.386173 25251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [term 1 LEADER]: Becoming Leader. State: Replica: 5d6a2003b3634857a51aa1766bcc791c, State: Running, Role: LEADER
I20260927 07:12:59.386170 25248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:12:59.386294 25251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d6a2003b3634857a51aa1766bcc791c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40635 } }
I20260927 07:12:59.386868 25252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d6a2003b3634857a51aa1766bcc791c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d6a2003b3634857a51aa1766bcc791c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40635 } } }
I20260927 07:12:59.386863 25253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d6a2003b3634857a51aa1766bcc791c. Latest consensus state: current_term: 1 leader_uuid: "5d6a2003b3634857a51aa1766bcc791c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d6a2003b3634857a51aa1766bcc791c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40635 } } }
I20260927 07:12:59.386968 25252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:59.386994 25253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c [sys.catalog]: This master's current role is: LEADER
I20260927 07:12:59.387176 25260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:12:59.387682 25260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:12:59.387802 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25180
I20260927 07:12:59.387917 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:12:59.388104 4793 external_mini_cluster.cc:1428] Setting key 9b4449e4d00437c0d771b3fd51c83d1e
I20260927 07:12:59.389199 25260 catalog_manager.cc:1347] Generated new cluster ID: 4467d56e0849468fb2420de52a97aa35
I20260927 07:12:59.389245 25260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:12:59.392165 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:59.388539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44137 (local address 127.0.0.1:40635)
0927 07:12:59.390715 (+ 2176us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:59.390725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:59.390759 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:12:59.390876 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:59.390883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:59.390913 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:59.391055 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:59.391658 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:59.391662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:59.391663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:59.391666 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:59.391706 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:59.391720 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:59.391788 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:59.391841 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 07:12:59.391980 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2124,"thread_start_us":61,"threads_started":1}
I20260927 07:12:59.393061 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:12:59.399657 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:12:59.394620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42945 (local address 127.0.0.1:40635)
0927 07:12:59.394680 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 07:12:59.394681 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:12:59.398776 (+ 4095us) server_negotiation.cc:407] Connection header received
0927 07:12:59.398840 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:12:59.398841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:12:59.398847 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:12:59.398894 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:12:59.399471 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:12:59.399472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:12:59.399472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:12:59.399475 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:12:59.399490 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:12:59.399494 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:12:59.399541 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 07:12:59.399582 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 07:12:59.399594 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:12:59.402454 25260 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:12:59.402637 25260 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:12:59.407019 25260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d6a2003b3634857a51aa1766bcc791c: Generated new TSK 0
I20260927 07:12:59.407191 25260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:12:59.408913 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25180
2026-09-27T07:12:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1195 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:12:59 dist-test-slave-zw7p krb5kdc[25279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:12:59 dist-test-slave-zw7p krb5kdc[25279](info): set up 2 sockets
Sep 27 07:12:59 dist-test-slave-zw7p krb5kdc[25279](info): commencing operation
krb5kdc: starting...
W20260927 07:13:00.350347 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:00 dist-test-slave-zw7p krb5kdc[25279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:00.461764 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:00.541572 25292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:00.541774 25292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:00.541795 25292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:00.542939 25292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:00.542979 25292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:00.542992 25292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:00.543004 25292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:00.543015 25292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:00.544409 25292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:00.544600 25292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:00.544780 25292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:00.549248 25297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:00.549256 25300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:00.549459 25292 server_base.cc:1034] running on GCE node
W20260927 07:13:00.549311 25298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:00.549702 25292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:00.549973 25292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:00.551113 25292 hybrid_clock.cc:648] HybridClock initialized: now 1790493180551099 us; error 31 us; skew 500 ppm
Sep 27 07:13:00 dist-test-slave-zw7p krb5kdc[25279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493180, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:00.557420 25292 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:00.557912 25292 webserver.cc:460] Webserver started at http://127.0.0.1:44529/ using document root <none> and password file <none>
I20260927 07:13:00.558148 25292 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:00.558197 25292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:00.558337 25292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:00.559563 25292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "98a714f79c434a839023aada33b4d06e"
format_stamp: "Formatted at 2026-09-27 07:13:00 on dist-test-slave-zw7p"
server_key: "443a6b28dc1cbc22778d47db234f8144"
server_key_iv: "11278ece594943cf48dce9e7ac27c647"
server_key_version: "encryptionkey@0"
I20260927 07:13:00.559859 25292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "98a714f79c434a839023aada33b4d06e"
format_stamp: "Formatted at 2026-09-27 07:13:00 on dist-test-slave-zw7p"
server_key: "443a6b28dc1cbc22778d47db234f8144"
server_key_iv: "11278ece594943cf48dce9e7ac27c647"
server_key_version: "encryptionkey@0"
I20260927 07:13:00.561033 25292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:13:00.561661 25307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:00.561789 25292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:00.561865 25292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "98a714f79c434a839023aada33b4d06e"
format_stamp: "Formatted at 2026-09-27 07:13:00 on dist-test-slave-zw7p"
server_key: "443a6b28dc1cbc22778d47db234f8144"
server_key_iv: "11278ece594943cf48dce9e7ac27c647"
server_key_version: "encryptionkey@0"
I20260927 07:13:00.561930 25292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:00.581735 25292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:00.583912 25292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:00.584029 25292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:00.587765 25292 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44521
I20260927 07:13:00.587823 25359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44521 every 8 connection(s)
I20260927 07:13:00.588133 25292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:00.589110 25360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:00.591423 25360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e: Bootstrap starting.
I20260927 07:13:00.591930 25360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:00.592151 25360 log.cc:824] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:00.592764 25360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e: No bootstrap required, opened a new log
I20260927 07:13:00.594276 25360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a714f79c434a839023aada33b4d06e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44521 } }
I20260927 07:13:00.594427 25360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:00.594448 25360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a714f79c434a839023aada33b4d06e, State: Initialized, Role: FOLLOWER
I20260927 07:13:00.594580 25360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a714f79c434a839023aada33b4d06e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44521 } }
I20260927 07:13:00.594655 25360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:00.594677 25360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:00.594712 25360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:00.595184 25360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a714f79c434a839023aada33b4d06e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44521 } }
I20260927 07:13:00.595268 25360 leader_election.cc:302] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98a714f79c434a839023aada33b4d06e; no voters:
I20260927 07:13:00.595414 25360 leader_election.cc:288] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:00.595463 25363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:00.595639 25360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:00.595640 25363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [term 1 LEADER]: Becoming Leader. State: Replica: 98a714f79c434a839023aada33b4d06e, State: Running, Role: LEADER
I20260927 07:13:00.595777 25363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a714f79c434a839023aada33b4d06e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44521 } }
I20260927 07:13:00.596405 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25292
I20260927 07:13:00.596370 25365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98a714f79c434a839023aada33b4d06e. Latest consensus state: current_term: 1 leader_uuid: "98a714f79c434a839023aada33b4d06e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a714f79c434a839023aada33b4d06e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44521 } } }
I20260927 07:13:00.596377 25364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98a714f79c434a839023aada33b4d06e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a714f79c434a839023aada33b4d06e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44521 } } }
I20260927 07:13:00.596454 25365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:00.596477 25364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:00.596510 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:00.596629 25372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:00.596694 4793 external_mini_cluster.cc:1428] Setting key 6e104102f63696085da76df10965ab6e
I20260927 07:13:00.597486 25372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:00.599133 25372 catalog_manager.cc:1347] Generated new cluster ID: eb09c4197c5a49d392434ce5a8081d73
I20260927 07:13:00.599184 25372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:13:00 dist-test-slave-zw7p krb5kdc[25279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 07:13:00.606401 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:00.597132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59675 (local address 127.0.0.1:44521)
0927 07:13:00.597263 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:00.597273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:00.597464 (+ 191us) server_negotiation.cc:407] Connection header received
0927 07:13:00.597680 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:00.597688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:00.597744 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:00.597861 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:00.598606 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:00.599687 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:00.600413 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:00.600828 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:00.600922 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:00.603085 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:00.603089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:00.603089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:00.603101 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:00.605044 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:00.605352 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:00.605355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:00.605355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:00.605453 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:00.605631 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:00.605632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:00.605632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:00.605863 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:00.606011 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:00.606104 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 07:13:00.606235 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":61,"threads_started":1}
I20260927 07:13:00.607208 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:00.613610 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:00.608437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56667 (local address 127.0.0.1:44521)
0927 07:13:00.608472 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:00.608474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:00.608582 (+ 108us) server_negotiation.cc:407] Connection header received
0927 07:13:00.608632 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:00.608633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:00.608643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:00.608686 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:00.609128 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:00.609610 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:00.610246 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:00.610446 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:00.610496 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:00.611522 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:00.611523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:00.611523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:00.611530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:00.612667 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:00.612955 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:00.612956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:00.612957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:00.613007 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:00.613161 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:00.613161 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:00.613162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:00.613261 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:00.613339 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:00.613459 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 07:13:00.613525 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 07:13:00.619115 25372 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:00.619617 25372 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:00.630201 25372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98a714f79c434a839023aada33b4d06e: Generated new TSK 0
I20260927 07:13:00.630422 25372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:00.640954 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25292
2026-09-27T07:13:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1232 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:00 dist-test-slave-zw7p krb5kdc[25391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:00 dist-test-slave-zw7p krb5kdc[25391](info): set up 2 sockets
Sep 27 07:13:00 dist-test-slave-zw7p krb5kdc[25391](info): commencing operation
krb5kdc: starting...
W20260927 07:13:01.594274 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:01 dist-test-slave-zw7p krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:01.705287 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39219
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:01.783423 25404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:01.783630 25404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:01.783660 25404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:01.784790 25404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:01.784832 25404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:01.784848 25404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:01.784864 25404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:01.784879 25404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:01.786290 25404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:01.786523 25404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:01.786749 25404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:01.791520 25409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:01.791531 25410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:01.791587 25404 server_base.cc:1034] running on GCE node
W20260927 07:13:01.791534 25412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:01.791913 25404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:01.792209 25404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:01.793346 25404 hybrid_clock.cc:648] HybridClock initialized: now 1790493181793332 us; error 31 us; skew 500 ppm
Sep 27 07:13:01 dist-test-slave-zw7p krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493181, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:01.795614 25404 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:01.796017 25404 webserver.cc:460] Webserver started at http://127.0.0.1:46783/ using document root <none> and password file <none>
I20260927 07:13:01.796206 25404 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:01.796248 25404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:01.796361 25404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:01.797346 25404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "b624e363c37d4af789dd2f7c6477a7f9"
format_stamp: "Formatted at 2026-09-27 07:13:01 on dist-test-slave-zw7p"
server_key: "a87be2a0d8a79281d5a2b96387777330"
server_key_iv: "76937301b90ec38e7da683572b5913db"
server_key_version: "encryptionkey@0"
I20260927 07:13:01.797653 25404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "b624e363c37d4af789dd2f7c6477a7f9"
format_stamp: "Formatted at 2026-09-27 07:13:01 on dist-test-slave-zw7p"
server_key: "a87be2a0d8a79281d5a2b96387777330"
server_key_iv: "76937301b90ec38e7da683572b5913db"
server_key_version: "encryptionkey@0"
I20260927 07:13:01.798839 25404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:13:01.799486 25419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:01.799610 25404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:13:01.799690 25404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "b624e363c37d4af789dd2f7c6477a7f9"
format_stamp: "Formatted at 2026-09-27 07:13:01 on dist-test-slave-zw7p"
server_key: "a87be2a0d8a79281d5a2b96387777330"
server_key_iv: "76937301b90ec38e7da683572b5913db"
server_key_version: "encryptionkey@0"
I20260927 07:13:01.799746 25404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:01.813657 25404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:01.815326 25404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:01.815457 25404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:01.819176 25404 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39219
I20260927 07:13:01.819204 25471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39219 every 8 connection(s)
I20260927 07:13:01.819536 25404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:01.820461 25472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:01.822950 25472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9: Bootstrap starting.
I20260927 07:13:01.823526 25472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:01.823752 25472 log.cc:824] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:01.824383 25472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9: No bootstrap required, opened a new log
I20260927 07:13:01.825881 25472 raft_consensus.cc:348] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b624e363c37d4af789dd2f7c6477a7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39219 } }
I20260927 07:13:01.826018 25472 raft_consensus.cc:374] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:01.826047 25472 raft_consensus.cc:729] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b624e363c37d4af789dd2f7c6477a7f9, State: Initialized, Role: FOLLOWER
I20260927 07:13:01.826159 25472 consensus_queue.cc:260] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b624e363c37d4af789dd2f7c6477a7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39219 } }
I20260927 07:13:01.826236 25472 raft_consensus.cc:388] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:01.826264 25472 raft_consensus.cc:482] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:01.826296 25472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:01.826790 25472 raft_consensus.cc:504] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b624e363c37d4af789dd2f7c6477a7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39219 } }
I20260927 07:13:01.826875 25472 leader_election.cc:302] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b624e363c37d4af789dd2f7c6477a7f9; no voters:
I20260927 07:13:01.826993 25472 leader_election.cc:288] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:01.827041 25475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:01.827205 25472 sys_catalog.cc:564] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:01.827220 25475 raft_consensus.cc:686] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [term 1 LEADER]: Becoming Leader. State: Replica: b624e363c37d4af789dd2f7c6477a7f9, State: Running, Role: LEADER
I20260927 07:13:01.827346 25475 consensus_queue.cc:237] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b624e363c37d4af789dd2f7c6477a7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39219 } }
I20260927 07:13:01.827688 25477 sys_catalog.cc:455] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b624e363c37d4af789dd2f7c6477a7f9. Latest consensus state: current_term: 1 leader_uuid: "b624e363c37d4af789dd2f7c6477a7f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b624e363c37d4af789dd2f7c6477a7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39219 } } }
I20260927 07:13:01.827700 25476 sys_catalog.cc:455] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b624e363c37d4af789dd2f7c6477a7f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b624e363c37d4af789dd2f7c6477a7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39219 } } }
I20260927 07:13:01.827834 25477 sys_catalog.cc:458] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:01.827916 25476 sys_catalog.cc:458] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:01.828248 25479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:01.828850 25479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:01.829523 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25404
I20260927 07:13:01.829620 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:01.829777 4793 external_mini_cluster.cc:1428] Setting key 8251c88af28db8abff889349ad5d591a
I20260927 07:13:01.830940 25479 catalog_manager.cc:1347] Generated new cluster ID: 300ad9ceb55c42439501a5c0304fc115
I20260927 07:13:01.830989 25479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:01.835870 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:01.830387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33933 (local address 127.0.0.1:39219)
0927 07:13:01.830529 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:01.830538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:01.830591 (+ 53us) server_negotiation.cc:407] Connection header received
0927 07:13:01.830693 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:01.830699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:01.830749 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:01.831156 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:01.831243 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:01.832597 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:01.833707 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:01.834315 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:01.834404 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:01.835124 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:01.835128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:01.835128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:01.835325 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:01.835362 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:01.835375 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:01.835432 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:01.835512 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 07:13:01.835651 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
I20260927 07:13:01.836714 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:01.841331 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:01.838051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40789 (local address 127.0.0.1:39219)
0927 07:13:01.838082 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:01.838084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:01.838254 (+ 170us) server_negotiation.cc:407] Connection header received
0927 07:13:01.838308 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:01.838309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:01.838318 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:01.838361 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:01.838480 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:01.839202 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:01.840191 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:01.840483 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:01.840531 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:01.841122 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:01.841123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:01.841123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:01.841129 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:01.841141 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:01.841145 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:01.841184 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:01.841237 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 07:13:01.841260 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 07:13:01.848021 25479 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:01.848554 25479 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:01.853508 25479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b624e363c37d4af789dd2f7c6477a7f9: Generated new TSK 0
I20260927 07:13:01.853672 25479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:01.860201 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25404
2026-09-27T07:13:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1219 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:01 dist-test-slave-zw7p krb5kdc[25503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:01 dist-test-slave-zw7p krb5kdc[25503](info): set up 2 sockets
Sep 27 07:13:01 dist-test-slave-zw7p krb5kdc[25503](info): commencing operation
krb5kdc: starting...
W20260927 07:13:02.802385 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:02 dist-test-slave-zw7p krb5kdc[25503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:02.913483 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:02.993371 25516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:02.993580 25516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:02.993602 25516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:02.994737 25516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:02.994772 25516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:02.994784 25516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:02.994796 25516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:02.994807 25516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:02.996292 25516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:02.996495 25516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:02.996670 25516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:03.001149 25521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:03.001264 25522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:03.001389 25516 server_base.cc:1034] running on GCE node
W20260927 07:13:03.001463 25524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:03.001688 25516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:03.001976 25516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:03.003111 25516 hybrid_clock.cc:648] HybridClock initialized: now 1790493183003096 us; error 36 us; skew 500 ppm
Sep 27 07:13:03 dist-test-slave-zw7p krb5kdc[25503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493183, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:03.005429 25516 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:03.005875 25516 webserver.cc:460] Webserver started at http://127.0.0.1:39169/ using document root <none> and password file <none>
I20260927 07:13:03.006094 25516 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:03.006143 25516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:03.006279 25516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:03.007359 25516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "87ce543769c14f1a98e834579980742d"
format_stamp: "Formatted at 2026-09-27 07:13:03 on dist-test-slave-zw7p"
server_key: "1f7b203f1775a0faf9d3a6bf9cb58c1d"
server_key_iv: "de750d63f7625cd456662806faa996cf"
server_key_version: "encryptionkey@0"
I20260927 07:13:03.007674 25516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "87ce543769c14f1a98e834579980742d"
format_stamp: "Formatted at 2026-09-27 07:13:03 on dist-test-slave-zw7p"
server_key: "1f7b203f1775a0faf9d3a6bf9cb58c1d"
server_key_iv: "de750d63f7625cd456662806faa996cf"
server_key_version: "encryptionkey@0"
I20260927 07:13:03.008865 25516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 07:13:03.009516 25531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:03.009671 25516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:13:03.009750 25516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "87ce543769c14f1a98e834579980742d"
format_stamp: "Formatted at 2026-09-27 07:13:03 on dist-test-slave-zw7p"
server_key: "1f7b203f1775a0faf9d3a6bf9cb58c1d"
server_key_iv: "de750d63f7625cd456662806faa996cf"
server_key_version: "encryptionkey@0"
I20260927 07:13:03.009809 25516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:03.012985 25516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:03.014639 25516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:03.014761 25516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:03.018358 25516 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37733
I20260927 07:13:03.018386 25583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37733 every 8 connection(s)
I20260927 07:13:03.018697 25516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:03.019589 25584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:03.021739 25584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d: Bootstrap starting.
I20260927 07:13:03.022275 25584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:03.022517 25584 log.cc:824] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:03.023137 25584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d: No bootstrap required, opened a new log
I20260927 07:13:03.024655 25584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ce543769c14f1a98e834579980742d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37733 } }
I20260927 07:13:03.024786 25584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:03.024818 25584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ce543769c14f1a98e834579980742d, State: Initialized, Role: FOLLOWER
I20260927 07:13:03.024950 25584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ce543769c14f1a98e834579980742d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37733 } }
I20260927 07:13:03.025007 25584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:03.025027 25584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:03.025071 25584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:03.025563 25584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ce543769c14f1a98e834579980742d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37733 } }
I20260927 07:13:03.025650 25584 leader_election.cc:302] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87ce543769c14f1a98e834579980742d; no voters:
I20260927 07:13:03.025802 25584 leader_election.cc:288] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:03.025843 25587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:03.025995 25587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [term 1 LEADER]: Becoming Leader. State: Replica: 87ce543769c14f1a98e834579980742d, State: Running, Role: LEADER
I20260927 07:13:03.026028 25584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:03.026084 25587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ce543769c14f1a98e834579980742d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37733 } }
I20260927 07:13:03.026450 25589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87ce543769c14f1a98e834579980742d. Latest consensus state: current_term: 1 leader_uuid: "87ce543769c14f1a98e834579980742d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ce543769c14f1a98e834579980742d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37733 } } }
I20260927 07:13:03.026526 25589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:03.026628 25588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87ce543769c14f1a98e834579980742d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ce543769c14f1a98e834579980742d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37733 } } }
I20260927 07:13:03.026680 25588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:03.026808 25596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:03.027315 25596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:03.027748 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25516
I20260927 07:13:03.027859 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:03.028053 4793 external_mini_cluster.cc:1428] Setting key 35510a153d5f8ad0d3f98c95b69fa637
I20260927 07:13:03.029043 25596 catalog_manager.cc:1347] Generated new cluster ID: 6f7e772035534a568ee4a9d3c49a9991
I20260927 07:13:03.029088 25596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:03.030674 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:03.028712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59695 (local address 127.0.0.1:37733)
0927 07:13:03.029112 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:03.029121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:03.029152 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:13:03.029267 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:03.029274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:03.029306 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:03.029448 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:03.030212 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:03.030216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:03.030216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:03.030221 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:03.030276 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:03.030291 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:03.030354 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:03.030416 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 07:13:03.030458 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":68,"threads_started":1}
I20260927 07:13:03.031556 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:03.034291 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:03.033040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49931 (local address 127.0.0.1:37733)
0927 07:13:03.033072 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:03.033074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:03.033288 (+ 214us) server_negotiation.cc:407] Connection header received
0927 07:13:03.033381 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:03.033381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:03.033386 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:03.033412 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:03.034062 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:03.034063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:03.034063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:03.034067 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:03.034082 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:03.034086 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:03.034115 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:03.034213 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 07:13:03.034227 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:13:03.041886 25596 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:03.042055 25596 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:03.047595 25596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87ce543769c14f1a98e834579980742d: Generated new TSK 0
I20260927 07:13:03.047798 25596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:03.056205 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25516
2026-09-27T07:13:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1196 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:03 dist-test-slave-zw7p krb5kdc[25615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:03 dist-test-slave-zw7p krb5kdc[25615](info): set up 2 sockets
Sep 27 07:13:03 dist-test-slave-zw7p krb5kdc[25615](info): commencing operation
krb5kdc: starting...
W20260927 07:13:03.998358 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:04 dist-test-slave-zw7p krb5kdc[25615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:04.107230 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41721
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:04.185313 25628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:04.185529 25628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:04.185552 25628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:04.186717 25628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:04.186753 25628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:04.186765 25628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:04.186777 25628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:04.186787 25628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:04.188215 25628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:04.188417 25628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:04.188594 25628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:04.193143 25636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:04.193264 25634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:04.193143 25633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:04.193248 25628 server_base.cc:1034] running on GCE node
I20260927 07:13:04.193557 25628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:04.193849 25628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:04.195029 25628 hybrid_clock.cc:648] HybridClock initialized: now 1790493184195011 us; error 29 us; skew 500 ppm
Sep 27 07:13:04 dist-test-slave-zw7p krb5kdc[25615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493184, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:04.197482 25628 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:04.197916 25628 webserver.cc:460] Webserver started at http://127.0.0.1:44861/ using document root <none> and password file <none>
I20260927 07:13:04.198140 25628 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:04.198187 25628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:04.198304 25628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:04.199406 25628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "14172c6e833642b2b8b547b108a265f0"
format_stamp: "Formatted at 2026-09-27 07:13:04 on dist-test-slave-zw7p"
server_key: "ec35380bd66e3a587da0b64398b54b06"
server_key_iv: "7fb397f5c3c8e077bc27b9b640c74ce1"
server_key_version: "encryptionkey@0"
I20260927 07:13:04.199736 25628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "14172c6e833642b2b8b547b108a265f0"
format_stamp: "Formatted at 2026-09-27 07:13:04 on dist-test-slave-zw7p"
server_key: "ec35380bd66e3a587da0b64398b54b06"
server_key_iv: "7fb397f5c3c8e077bc27b9b640c74ce1"
server_key_version: "encryptionkey@0"
I20260927 07:13:04.200968 25628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 07:13:04.201638 25643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:04.201803 25628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:13:04.201877 25628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "14172c6e833642b2b8b547b108a265f0"
format_stamp: "Formatted at 2026-09-27 07:13:04 on dist-test-slave-zw7p"
server_key: "ec35380bd66e3a587da0b64398b54b06"
server_key_iv: "7fb397f5c3c8e077bc27b9b640c74ce1"
server_key_version: "encryptionkey@0"
I20260927 07:13:04.201936 25628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:04.223889 25628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:04.225729 25628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:04.225878 25628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:04.230010 25628 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41721
I20260927 07:13:04.230046 25695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41721 every 8 connection(s)
I20260927 07:13:04.230401 25628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:04.231318 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25628
I20260927 07:13:04.231406 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:04.231390 25696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:04.231583 4793 external_mini_cluster.cc:1428] Setting key c61f1221fc441072578a9c69b29f612c
I20260927 07:13:04.234323 25696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0: Bootstrap starting.
I20260927 07:13:04.235025 25696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:04.235338 25696 log.cc:824] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:04.236033 25696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0: No bootstrap required, opened a new log
Sep 27 07:13:04 dist-test-slave-zw7p krb5kdc[25615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 07:13:04.238314 25696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14172c6e833642b2b8b547b108a265f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41721 } }
I20260927 07:13:04.238472 25696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:04.238504 25696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14172c6e833642b2b8b547b108a265f0, State: Initialized, Role: FOLLOWER
I20260927 07:13:04.238654 25696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14172c6e833642b2b8b547b108a265f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41721 } }
I20260927 07:13:04.238739 25696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:04.238770 25696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:04.238817 25696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:04.239336 25696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14172c6e833642b2b8b547b108a265f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41721 } }
I20260927 07:13:04.239434 25696 leader_election.cc:302] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14172c6e833642b2b8b547b108a265f0; no voters:
I20260927 07:13:04.239607 25696 leader_election.cc:288] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:04.239663 25701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:04.239835 25701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [term 1 LEADER]: Becoming Leader. State: Replica: 14172c6e833642b2b8b547b108a265f0, State: Running, Role: LEADER
I20260927 07:13:04.239943 25696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:04.239979 25701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14172c6e833642b2b8b547b108a265f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41721 } }
I20260927 07:13:04.240329 25701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14172c6e833642b2b8b547b108a265f0. Latest consensus state: current_term: 1 leader_uuid: "14172c6e833642b2b8b547b108a265f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14172c6e833642b2b8b547b108a265f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41721 } } }
I20260927 07:13:04.240528 25701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:04.240470 25703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14172c6e833642b2b8b547b108a265f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14172c6e833642b2b8b547b108a265f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41721 } } }
I20260927 07:13:04.240778 25703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:04.241158 25710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:04.241855 25710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:04.242234 25699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:04.232083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:34269 (local address 127.0.0.1:41721)
0927 07:13:04.232231 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:04.232239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:04.232264 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:13:04.232741 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:04.232748 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:04.232800 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:04.232917 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:04.233667 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:04.234698 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:04.235711 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:04.236171 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:04.236259 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:04.238748 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:04.238752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:04.238752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:04.238763 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:04.240615 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:04.241072 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:04.241075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:04.241076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:04.241195 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:04.241445 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:04.241446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:04.241446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:04.241677 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:04.241842 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:04.241968 (+ 126us) server_negotiation.cc:299] Negotiation successful
0927 07:13:04.242067 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":53,"threads_started":1}
I20260927 07:13:04.243170 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:04.244872 25710 catalog_manager.cc:1347] Generated new cluster ID: 10b877a0da7d49bfa1bd5d5b14ea61cf
I20260927 07:13:04.244925 25710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:04.250200 25699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:04.244649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:59045 (local address 127.0.0.1:41721)
0927 07:13:04.244682 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:04.244684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:04.244826 (+ 142us) server_negotiation.cc:407] Connection header received
0927 07:13:04.244881 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:04.244882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:04.244892 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:04.244939 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:04.245447 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:04.245943 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:04.246668 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:04.246889 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:04.246946 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:04.248017 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:04.248019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:04.248019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:04.248026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:04.249202 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:04.249522 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:04.249523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:04.249523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:04.249571 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:04.249753 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:04.249754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:04.249754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:04.249855 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:04.249942 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:04.250057 (+ 115us) server_negotiation.cc:299] Negotiation successful
0927 07:13:04.250115 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:13:04.258280 25710 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:04.259047 25710 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:04.280920 25710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14172c6e833642b2b8b547b108a265f0: Generated new TSK 0
I20260927 07:13:04.281163 25710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:04.291067 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25628
W20260927 07:13:04.295122 25613 connection.cc:535] client connection to 127.0.0.1:41721 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41721): Connection reset by peer (error 104)
2026-09-27T07:13:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1235 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:04 dist-test-slave-zw7p krb5kdc[25727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:04 dist-test-slave-zw7p krb5kdc[25727](info): set up 2 sockets
Sep 27 07:13:04 dist-test-slave-zw7p krb5kdc[25727](info): commencing operation
krb5kdc: starting...
W20260927 07:13:05.238348 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:05 dist-test-slave-zw7p krb5kdc[25727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:05.348730 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:05.427788 25740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:05.428000 25740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:05.428032 25740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:05.429193 25740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:05.429246 25740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:05.429265 25740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:05.429280 25740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:05.429296 25740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:05.430863 25740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:05.431118 25740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:05.431336 25740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 07:13:05.436025 25740 server_base.cc:1034] running on GCE node
W20260927 07:13:05.436014 25748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:05.435953 25746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:05.435957 25745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:05.436486 25740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:05.436802 25740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:05.437963 25740 hybrid_clock.cc:648] HybridClock initialized: now 1790493185437948 us; error 27 us; skew 500 ppm
Sep 27 07:13:05 dist-test-slave-zw7p krb5kdc[25727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493185, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:05.440332 25740 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:05.440730 25740 webserver.cc:460] Webserver started at http://127.0.0.1:43107/ using document root <none> and password file <none>
I20260927 07:13:05.440944 25740 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:05.440994 25740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:05.441116 25740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:05.442148 25740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "6672eb95985a40e8abf58ca12d8686f1"
format_stamp: "Formatted at 2026-09-27 07:13:05 on dist-test-slave-zw7p"
server_key: "d1a96bf7c1c0ed25c5fa2551cfe8c2f0"
server_key_iv: "f5dfa1083498227fc96dfb4989d4b952"
server_key_version: "encryptionkey@0"
I20260927 07:13:05.442487 25740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "6672eb95985a40e8abf58ca12d8686f1"
format_stamp: "Formatted at 2026-09-27 07:13:05 on dist-test-slave-zw7p"
server_key: "d1a96bf7c1c0ed25c5fa2551cfe8c2f0"
server_key_iv: "f5dfa1083498227fc96dfb4989d4b952"
server_key_version: "encryptionkey@0"
I20260927 07:13:05.443728 25740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:13:05.444403 25755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:05.444551 25740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:13:05.444633 25740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "6672eb95985a40e8abf58ca12d8686f1"
format_stamp: "Formatted at 2026-09-27 07:13:05 on dist-test-slave-zw7p"
server_key: "d1a96bf7c1c0ed25c5fa2551cfe8c2f0"
server_key_iv: "f5dfa1083498227fc96dfb4989d4b952"
server_key_version: "encryptionkey@0"
I20260927 07:13:05.444694 25740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:05.458068 25740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:05.459765 25740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:05.459897 25740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:05.463531 25740 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46137
I20260927 07:13:05.463567 25807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46137 every 8 connection(s)
I20260927 07:13:05.463889 25740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:05.464815 25808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:05.467130 25808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1: Bootstrap starting.
I20260927 07:13:05.467686 25808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:05.467931 25808 log.cc:824] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:05.468577 25808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1: No bootstrap required, opened a new log
I20260927 07:13:05.470100 25808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672eb95985a40e8abf58ca12d8686f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46137 } }
I20260927 07:13:05.470256 25808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:05.470279 25808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6672eb95985a40e8abf58ca12d8686f1, State: Initialized, Role: FOLLOWER
I20260927 07:13:05.470425 25808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672eb95985a40e8abf58ca12d8686f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46137 } }
I20260927 07:13:05.470487 25808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:05.470510 25808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:05.470585 25808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:05.471079 25808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672eb95985a40e8abf58ca12d8686f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46137 } }
I20260927 07:13:05.471170 25808 leader_election.cc:302] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6672eb95985a40e8abf58ca12d8686f1; no voters:
I20260927 07:13:05.471319 25808 leader_election.cc:288] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:05.471378 25811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:05.471582 25811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [term 1 LEADER]: Becoming Leader. State: Replica: 6672eb95985a40e8abf58ca12d8686f1, State: Running, Role: LEADER
I20260927 07:13:05.471567 25808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:05.471743 25811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672eb95985a40e8abf58ca12d8686f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46137 } }
I20260927 07:13:05.472142 25812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6672eb95985a40e8abf58ca12d8686f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672eb95985a40e8abf58ca12d8686f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46137 } } }
I20260927 07:13:05.472141 25813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6672eb95985a40e8abf58ca12d8686f1. Latest consensus state: current_term: 1 leader_uuid: "6672eb95985a40e8abf58ca12d8686f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672eb95985a40e8abf58ca12d8686f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46137 } } }
I20260927 07:13:05.472287 25812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:05.472362 25813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:05.473340 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25740
I20260927 07:13:05.473464 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:05.473670 4793 external_mini_cluster.cc:1428] Setting key fb8341ddebeac70fefd00f7be5c2e8da
W20260927 07:13:05.473668 25826 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:05.474313 25826 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:13:05.474401 25827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:05.474622 25827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:05.476214 25827 catalog_manager.cc:1347] Generated new cluster ID: a1a0c12671dc4a6fb5cdce21965b1dd9
I20260927 07:13:05.476274 25827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:05.478843 25830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:05.474124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:40741 (local address 127.0.0.1:46137)
0927 07:13:05.474421 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:05.474431 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:05.474466 (+ 35us) server_negotiation.cc:407] Connection header received
0927 07:13:05.474614 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:05.474621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:05.474691 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:05.474830 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:05.475027 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:05.476312 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:05.477054 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:05.477461 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:05.477551 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:05.478173 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:05.478177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:05.478178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:05.478340 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:05.478370 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:05.478381 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:05.478450 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:05.478539 (+ 89us) server_negotiation.cc:299] Negotiation successful
0927 07:13:05.478675 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260927 07:13:05.479696 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:05.483585 25830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:05.480948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:59697 (local address 127.0.0.1:46137)
0927 07:13:05.480981 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:05.480982 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:05.481075 (+ 93us) server_negotiation.cc:407] Connection header received
0927 07:13:05.481126 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:05.481127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:05.481137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:05.481181 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:05.481319 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:05.481781 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:05.482449 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:05.482726 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:05.482781 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:05.483369 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:05.483370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:05.483371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:05.483375 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:05.483389 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:05.483394 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:05.483441 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:05.483502 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 07:13:05.483514 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:13:05.500869 25827 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:05.501434 25827 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:05.505944 25827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6672eb95985a40e8abf58ca12d8686f1: Generated new TSK 0
I20260927 07:13:05.506124 25827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:05.509866 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25740
W20260927 07:13:05.513904 25725 connection.cc:535] client connection to 127.0.0.1:46137 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46137): Connection reset by peer (error 104)
2026-09-27T07:13:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1223 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:05 dist-test-slave-zw7p krb5kdc[25839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:05 dist-test-slave-zw7p krb5kdc[25839](info): set up 2 sockets
Sep 27 07:13:05 dist-test-slave-zw7p krb5kdc[25839](info): commencing operation
krb5kdc: starting...
W20260927 07:13:06.462337 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:06 dist-test-slave-zw7p krb5kdc[25839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:06.572945 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:06.652697 25852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:06.652904 25852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:06.652935 25852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:06.654098 25852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:06.654139 25852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:06.654157 25852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:06.654172 25852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:06.654187 25852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:06.655689 25852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:06.655941 25852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:06.656159 25852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:06.660707 25860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:06.660794 25857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:06.660800 25858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:06.661067 25852 server_base.cc:1034] running on GCE node
I20260927 07:13:06.661300 25852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:06.661573 25852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:06.662708 25852 hybrid_clock.cc:648] HybridClock initialized: now 1790493186662690 us; error 36 us; skew 500 ppm
Sep 27 07:13:06 dist-test-slave-zw7p krb5kdc[25839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493186, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:06.674207 25852 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:06.674763 25852 webserver.cc:460] Webserver started at http://127.0.0.1:36199/ using document root <none> and password file <none>
I20260927 07:13:06.674996 25852 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:06.675072 25852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:06.675208 25852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:06.676313 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5"
format_stamp: "Formatted at 2026-09-27 07:13:06 on dist-test-slave-zw7p"
server_key: "46f7fb6cee4ff7f3c3dbacfe3de4a18d"
server_key_iv: "f21d5e51dffd03af86070cbe9eec63f1"
server_key_version: "encryptionkey@0"
I20260927 07:13:06.676661 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5"
format_stamp: "Formatted at 2026-09-27 07:13:06 on dist-test-slave-zw7p"
server_key: "46f7fb6cee4ff7f3c3dbacfe3de4a18d"
server_key_iv: "f21d5e51dffd03af86070cbe9eec63f1"
server_key_version: "encryptionkey@0"
I20260927 07:13:06.678009 25852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:13:06.678884 25867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:06.679067 25852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:13:06.679208 25852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5"
format_stamp: "Formatted at 2026-09-27 07:13:06 on dist-test-slave-zw7p"
server_key: "46f7fb6cee4ff7f3c3dbacfe3de4a18d"
server_key_iv: "f21d5e51dffd03af86070cbe9eec63f1"
server_key_version: "encryptionkey@0"
I20260927 07:13:06.679275 25852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:06.682706 25852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:06.684710 25852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:06.684856 25852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:06.688673 25852 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46261
I20260927 07:13:06.688714 25919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46261 every 8 connection(s)
I20260927 07:13:06.689033 25852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:06.689986 25920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:06.692312 25920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5: Bootstrap starting.
I20260927 07:13:06.692853 25920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:06.693100 25920 log.cc:824] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:06.693775 25920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5: No bootstrap required, opened a new log
I20260927 07:13:06.695405 25920 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } }
I20260927 07:13:06.695544 25920 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:06.695564 25920 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6e2c4ffca4f7d8db37aa2ce057bb5, State: Initialized, Role: FOLLOWER
I20260927 07:13:06.695699 25920 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } }
I20260927 07:13:06.695771 25920 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:06.695803 25920 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:06.695850 25920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:06.696393 25920 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } }
I20260927 07:13:06.696487 25920 leader_election.cc:302] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4f6e2c4ffca4f7d8db37aa2ce057bb5; no voters:
I20260927 07:13:06.696640 25920 leader_election.cc:288] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:06.696679 25923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:06.696825 25923 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6e2c4ffca4f7d8db37aa2ce057bb5, State: Running, Role: LEADER
I20260927 07:13:06.696887 25920 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:06.696924 25923 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } }
I20260927 07:13:06.697279 25925 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4f6e2c4ffca4f7d8db37aa2ce057bb5. Latest consensus state: current_term: 1 leader_uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } } }
I20260927 07:13:06.697360 25925 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:06.697631 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25852
I20260927 07:13:06.697742 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:06.697923 4793 external_mini_cluster.cc:1428] Setting key 6cddd146c465ddd9e9f186d417ce8ba7
I20260927 07:13:06.697568 25931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:06.697425 25924 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6e2c4ffca4f7d8db37aa2ce057bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } } }
I20260927 07:13:06.698518 25924 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:06.698647 25931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:06.700114 25931 catalog_manager.cc:1347] Generated new cluster ID: ddc3f475aefd42cda9940ed4d72377d8
I20260927 07:13:06.700163 25931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:06.704082 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:06.698395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:36925 (local address 127.0.0.1:46261)
0927 07:13:06.702670 (+ 4275us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:06.702679 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:06.702712 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:13:06.702821 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:06.702827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:06.702865 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:06.702998 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:06.703688 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:06.703691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:06.703692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:06.703696 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:06.703733 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:06.703746 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:06.703793 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:06.703864 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 07:13:06.703905 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4213,"thread_start_us":74,"threads_started":1}
I20260927 07:13:06.705205 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:06.707795 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:06.706639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:41895 (local address 127.0.0.1:46261)
0927 07:13:06.706686 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:06.706688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:06.706907 (+ 219us) server_negotiation.cc:407] Connection header received
0927 07:13:06.706949 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:06.706950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:06.706956 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:06.707018 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:06.707582 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:06.707583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:06.707583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:06.707588 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:06.707612 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:06.707616 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:06.707673 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:06.707716 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 07:13:06.707729 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 07:13:06.713953 25931 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:06.714107 25931 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:06.721361 25931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4f6e2c4ffca4f7d8db37aa2ce057bb5: Generated new TSK 0
I20260927 07:13:06.721550 25931 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:06.730856 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25852
2026-09-27T07:13:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1216 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:06 dist-test-slave-zw7p krb5kdc[25951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:06 dist-test-slave-zw7p krb5kdc[25951](info): set up 2 sockets
Sep 27 07:13:06 dist-test-slave-zw7p krb5kdc[25951](info): commencing operation
krb5kdc: starting...
W20260927 07:13:07.678483 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:07 dist-test-slave-zw7p krb5kdc[25951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:07.793100 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:07.872584 25964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:07.872807 25964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:07.872829 25964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:07.874025 25964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:07.874073 25964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:07.874086 25964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:07.874102 25964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:07.874125 25964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:07.875699 25964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:07.875959 25964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:07.876184 25964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:07.880837 25969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:07.880815 25972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:07.880892 25970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:07.881086 25964 server_base.cc:1034] running on GCE node
I20260927 07:13:07.881305 25964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:07.881572 25964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:07.882732 25964 hybrid_clock.cc:648] HybridClock initialized: now 1790493187882719 us; error 27 us; skew 500 ppm
Sep 27 07:13:07 dist-test-slave-zw7p krb5kdc[25951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493187, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:07.885082 25964 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:07.885515 25964 webserver.cc:460] Webserver started at http://127.0.0.1:41331/ using document root <none> and password file <none>
I20260927 07:13:07.885746 25964 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:07.885794 25964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:07.885918 25964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:07.887015 25964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e"
format_stamp: "Formatted at 2026-09-27 07:13:07 on dist-test-slave-zw7p"
server_key: "4e6fd10a455a006647616f85741d702e"
server_key_iv: "d5e589c6c642da9befab2e471c12d388"
server_key_version: "encryptionkey@0"
I20260927 07:13:07.887344 25964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e"
format_stamp: "Formatted at 2026-09-27 07:13:07 on dist-test-slave-zw7p"
server_key: "4e6fd10a455a006647616f85741d702e"
server_key_iv: "d5e589c6c642da9befab2e471c12d388"
server_key_version: "encryptionkey@0"
I20260927 07:13:07.888525 25964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:13:07.889149 25979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:07.889297 25964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:07.889385 25964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e"
format_stamp: "Formatted at 2026-09-27 07:13:07 on dist-test-slave-zw7p"
server_key: "4e6fd10a455a006647616f85741d702e"
server_key_iv: "d5e589c6c642da9befab2e471c12d388"
server_key_version: "encryptionkey@0"
I20260927 07:13:07.889451 25964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:07.920595 25964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:07.922323 25964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:07.922461 25964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:07.926208 25964 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46333
I20260927 07:13:07.926265 26031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46333 every 8 connection(s)
I20260927 07:13:07.926599 25964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:07.927521 26032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:07.928092 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 25964
I20260927 07:13:07.928210 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:07.928383 4793 external_mini_cluster.cc:1428] Setting key 6445fb206f702a4c6d4b45af5e375a04
I20260927 07:13:07.930286 26032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e: Bootstrap starting.
I20260927 07:13:07.930895 26032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:07.931155 26032 log.cc:824] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:07.931872 26032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e: No bootstrap required, opened a new log
I20260927 07:13:07.934068 26032 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46333 } }
I20260927 07:13:07.934237 26032 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:07.934285 26032 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfec3ad7b3374453bfe3f0da32dbdd0e, State: Initialized, Role: FOLLOWER
I20260927 07:13:07.934417 26032 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46333 } }
I20260927 07:13:07.934480 26032 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:07.934500 26032 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:07.934531 26032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 0 FOLLOWER]: Advancing to term 1
Sep 27 07:13:07 dist-test-slave-zw7p krb5kdc[25951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 07:13:07.935058 26032 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46333 } }
I20260927 07:13:07.935166 26032 leader_election.cc:302] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfec3ad7b3374453bfe3f0da32dbdd0e; no voters:
I20260927 07:13:07.935328 26032 leader_election.cc:288] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:07.935369 26037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:07.935503 26037 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [term 1 LEADER]: Becoming Leader. State: Replica: bfec3ad7b3374453bfe3f0da32dbdd0e, State: Running, Role: LEADER
I20260927 07:13:07.935581 26037 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46333 } }
I20260927 07:13:07.935582 26032 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:07.935927 26037 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfec3ad7b3374453bfe3f0da32dbdd0e. Latest consensus state: current_term: 1 leader_uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46333 } } }
I20260927 07:13:07.935995 26038 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfec3ad7b3374453bfe3f0da32dbdd0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46333 } } }
I20260927 07:13:07.936055 26037 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:07.936543 26038 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e [sys.catalog]: This master's current role is: LEADER
W20260927 07:13:07.937366 26053 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:07.937408 26053 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:13:07.937450 26044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:07.937623 26044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:07.938941 26044 catalog_manager.cc:1347] Generated new cluster ID: b6aa2f806e4d4e99a2d480f8c41468ae
I20260927 07:13:07.938985 26044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:07.939529 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:07.928883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:37767 (local address 127.0.0.1:46333)
0927 07:13:07.929030 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:07.929040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:07.929067 (+ 27us) server_negotiation.cc:407] Connection header received
0927 07:13:07.929459 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:07.929465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:07.929512 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:07.929636 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:07.930351 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:07.931371 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:07.932320 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:07.932752 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:07.932833 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:07.935436 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:07.935441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:07.935441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:07.935453 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:07.937458 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:07.937892 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:07.937895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:07.937896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:07.938001 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:07.938253 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:07.938254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:07.938255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:07.938588 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:07.938731 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:07.939230 (+ 499us) server_negotiation.cc:299] Negotiation successful
0927 07:13:07.939333 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260927 07:13:07.940431 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:07.949976 26044 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:07.950450 26044 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:07.957603 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:07.942930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:44761 (local address 127.0.0.1:46333)
0927 07:13:07.942962 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:07.942964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:07.949741 (+ 6777us) server_negotiation.cc:407] Connection header received
0927 07:13:07.949802 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:07.949804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:07.949814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:07.949874 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:07.951112 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:07.951871 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:07.952808 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:07.953529 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:07.953599 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:07.954922 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:07.954923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:07.954923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:07.954930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:07.956440 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:07.956852 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:07.956853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:07.956854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:07.956913 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:07.957106 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:07.957107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:07.957107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:07.957227 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:07.957312 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:07.957450 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 07:13:07.957513 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":16}
I20260927 07:13:07.963842 26044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfec3ad7b3374453bfe3f0da32dbdd0e: Generated new TSK 0
I20260927 07:13:07.964032 26044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:07.966871 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 25964
W20260927 07:13:07.970853 25949 connection.cc:535] client connection to 127.0.0.1:46333 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46333): Connection reset by peer (error 104)
2026-09-27T07:13:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1236 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:08 dist-test-slave-zw7p krb5kdc[26063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:08 dist-test-slave-zw7p krb5kdc[26063](info): set up 2 sockets
Sep 27 07:13:08 dist-test-slave-zw7p krb5kdc[26063](info): commencing operation
krb5kdc: starting...
W20260927 07:13:08.910416 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:08 dist-test-slave-zw7p krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:09.021210 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:09.098732 26076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:09.098932 26076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:09.098960 26076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:09.100098 26076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:09.100138 26076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:09.100157 26076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:09.100171 26076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:09.100185 26076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:09.101635 26076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40811
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:09.101876 26076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:09.102084 26076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:09.106590 26081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:09.106634 26084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:09.106604 26082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:09.106894 26076 server_base.cc:1034] running on GCE node
I20260927 07:13:09.107121 26076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:09.107380 26076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:09.108517 26076 hybrid_clock.cc:648] HybridClock initialized: now 1790493189108502 us; error 28 us; skew 500 ppm
Sep 27 07:13:09 dist-test-slave-zw7p krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493189, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:09.110944 26076 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:09.111326 26076 webserver.cc:460] Webserver started at http://127.0.0.1:35571/ using document root <none> and password file <none>
I20260927 07:13:09.111493 26076 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:09.111572 26076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:09.111665 26076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:09.112627 26076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "91fb3e2cae6e40759ef860caf4e70d36"
format_stamp: "Formatted at 2026-09-27 07:13:09 on dist-test-slave-zw7p"
server_key: "9c6f8be532d16c20ffa0ce068e90d7af"
server_key_iv: "2cd9601c72f157bdd441a8f5c5a97bc6"
server_key_version: "encryptionkey@0"
I20260927 07:13:09.112905 26076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "91fb3e2cae6e40759ef860caf4e70d36"
format_stamp: "Formatted at 2026-09-27 07:13:09 on dist-test-slave-zw7p"
server_key: "9c6f8be532d16c20ffa0ce068e90d7af"
server_key_iv: "2cd9601c72f157bdd441a8f5c5a97bc6"
server_key_version: "encryptionkey@0"
I20260927 07:13:09.114012 26076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 07:13:09.114647 26091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:09.114809 26076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:09.114907 26076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "91fb3e2cae6e40759ef860caf4e70d36"
format_stamp: "Formatted at 2026-09-27 07:13:09 on dist-test-slave-zw7p"
server_key: "9c6f8be532d16c20ffa0ce068e90d7af"
server_key_iv: "2cd9601c72f157bdd441a8f5c5a97bc6"
server_key_version: "encryptionkey@0"
I20260927 07:13:09.114965 26076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:09.130462 26076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:09.132560 26076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:09.132696 26076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:09.136358 26076 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40811
I20260927 07:13:09.136410 26143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40811 every 8 connection(s)
I20260927 07:13:09.136713 26076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:09.137526 26144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:09.139726 26144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36: Bootstrap starting.
I20260927 07:13:09.140236 26144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:09.140455 26144 log.cc:824] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:09.141057 26144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36: No bootstrap required, opened a new log
I20260927 07:13:09.142520 26144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb3e2cae6e40759ef860caf4e70d36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40811 } }
I20260927 07:13:09.142701 26144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:09.142732 26144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91fb3e2cae6e40759ef860caf4e70d36, State: Initialized, Role: FOLLOWER
I20260927 07:13:09.142839 26144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb3e2cae6e40759ef860caf4e70d36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40811 } }
I20260927 07:13:09.142912 26144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:09.142939 26144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:09.142971 26144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:09.143417 26144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb3e2cae6e40759ef860caf4e70d36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40811 } }
I20260927 07:13:09.143498 26144 leader_election.cc:302] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91fb3e2cae6e40759ef860caf4e70d36; no voters:
I20260927 07:13:09.143616 26144 leader_election.cc:288] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:09.143662 26147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:09.143815 26144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:09.143828 26147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [term 1 LEADER]: Becoming Leader. State: Replica: 91fb3e2cae6e40759ef860caf4e70d36, State: Running, Role: LEADER
I20260927 07:13:09.143909 26147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb3e2cae6e40759ef860caf4e70d36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40811 } }
I20260927 07:13:09.144418 26148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91fb3e2cae6e40759ef860caf4e70d36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb3e2cae6e40759ef860caf4e70d36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40811 } } }
I20260927 07:13:09.144513 26148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:09.144658 26149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91fb3e2cae6e40759ef860caf4e70d36. Latest consensus state: current_term: 1 leader_uuid: "91fb3e2cae6e40759ef860caf4e70d36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb3e2cae6e40759ef860caf4e70d36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40811 } } }
I20260927 07:13:09.144712 26149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:09.144791 26156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:09.145205 26156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:09.145397 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26076
I20260927 07:13:09.145488 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:09.145655 4793 external_mini_cluster.cc:1428] Setting key b645a1cf18fb460ad58ae42ca4bafd85
I20260927 07:13:09.146988 26156 catalog_manager.cc:1347] Generated new cluster ID: dded0b48796440bfa3137fb65775cbf5
I20260927 07:13:09.147042 26156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:09.155807 26166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:09.146273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:37439 (local address 127.0.0.1:40811)
0927 07:13:09.146432 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:09.146442 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:09.146472 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:13:09.146612 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:09.146619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:09.146670 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:09.146781 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:09.150665 (+ 3884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:09.152397 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:09.153540 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:09.154239 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:09.154357 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:09.155054 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:09.155059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:09.155059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:09.155282 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:09.155323 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:09.155339 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:09.155418 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:09.155509 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 07:13:09.155540 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260927 07:13:09.156766 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:09.159941 26156 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:09.160465 26156 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:09.161515 26166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:09.158156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:51695 (local address 127.0.0.1:40811)
0927 07:13:09.158189 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:09.158190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:09.158311 (+ 121us) server_negotiation.cc:407] Connection header received
0927 07:13:09.158372 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:09.158373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:09.158383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:09.158437 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:09.158601 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:09.159313 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:09.160329 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:09.160621 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:09.160682 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:09.161292 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:09.161294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:09.161294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:09.161298 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:09.161312 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:09.161316 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:09.161365 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:09.161432 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 07:13:09.161443 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:13:09.166142 26156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91fb3e2cae6e40759ef860caf4e70d36: Generated new TSK 0
I20260927 07:13:09.166307 26156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:09.169072 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26076
W20260927 07:13:09.172829 26061 connection.cc:535] client connection to 127.0.0.1:40811 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40811): Connection reset by peer (error 104)
2026-09-27T07:13:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1202 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:09 dist-test-slave-zw7p krb5kdc[26175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:09 dist-test-slave-zw7p krb5kdc[26175](info): set up 2 sockets
Sep 27 07:13:09 dist-test-slave-zw7p krb5kdc[26175](info): commencing operation
krb5kdc: starting...
W20260927 07:13:10.122394 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:10 dist-test-slave-zw7p krb5kdc[26175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:13:10.235222 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:10.314471 26188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:10.314736 26188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:10.314769 26188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:10.315930 26188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:10.315969 26188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:10.315989 26188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:10.316011 26188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:10.316028 26188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:10.317472 26188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:10.317709 26188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:10.317909 26188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:10.322311 26196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:10.322326 26194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:10.322479 26193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:10.322682 26188 server_base.cc:1034] running on GCE node
I20260927 07:13:10.322877 26188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:10.323117 26188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:10.324267 26188 hybrid_clock.cc:648] HybridClock initialized: now 1790493190324248 us; error 33 us; skew 500 ppm
Sep 27 07:13:10 dist-test-slave-zw7p krb5kdc[26175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493190, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:10.326613 26188 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:13:10.327013 26188 webserver.cc:460] Webserver started at http://127.0.0.1:35689/ using document root <none> and password file <none>
I20260927 07:13:10.327206 26188 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:10.327248 26188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:10.327361 26188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:10.328369 26188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "f94add5173a444399e36c75e764f9837"
format_stamp: "Formatted at 2026-09-27 07:13:10 on dist-test-slave-zw7p"
server_key: "ebd241414bbefe878cf0fe79de8a5b77"
server_key_iv: "c51ebd00c9aca59de8365a236f86a1d4"
server_key_version: "encryptionkey@0"
I20260927 07:13:10.328684 26188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "f94add5173a444399e36c75e764f9837"
format_stamp: "Formatted at 2026-09-27 07:13:10 on dist-test-slave-zw7p"
server_key: "ebd241414bbefe878cf0fe79de8a5b77"
server_key_iv: "c51ebd00c9aca59de8365a236f86a1d4"
server_key_version: "encryptionkey@0"
I20260927 07:13:10.329912 26188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:13:10.330675 26203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:10.330839 26188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 07:13:10.330922 26188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "f94add5173a444399e36c75e764f9837"
format_stamp: "Formatted at 2026-09-27 07:13:10 on dist-test-slave-zw7p"
server_key: "ebd241414bbefe878cf0fe79de8a5b77"
server_key_iv: "c51ebd00c9aca59de8365a236f86a1d4"
server_key_version: "encryptionkey@0"
I20260927 07:13:10.330987 26188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:10.334172 26188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:10.336225 26188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:10.336336 26188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:10.340065 26188 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35545
I20260927 07:13:10.340101 26255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35545 every 8 connection(s)
I20260927 07:13:10.340397 26188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:10.341349 26256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:10.343639 26256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837: Bootstrap starting.
I20260927 07:13:10.344168 26256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:10.344384 26256 log.cc:824] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:10.344983 26256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837: No bootstrap required, opened a new log
I20260927 07:13:10.346488 26256 raft_consensus.cc:348] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94add5173a444399e36c75e764f9837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260927 07:13:10.346678 26256 raft_consensus.cc:374] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:10.346706 26256 raft_consensus.cc:729] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f94add5173a444399e36c75e764f9837, State: Initialized, Role: FOLLOWER
I20260927 07:13:10.346808 26256 consensus_queue.cc:260] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94add5173a444399e36c75e764f9837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260927 07:13:10.346858 26256 raft_consensus.cc:388] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:10.346882 26256 raft_consensus.cc:482] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:10.346912 26256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:10.347376 26256 raft_consensus.cc:504] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94add5173a444399e36c75e764f9837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260927 07:13:10.347457 26256 leader_election.cc:302] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f94add5173a444399e36c75e764f9837; no voters:
I20260927 07:13:10.347573 26256 leader_election.cc:288] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:10.347623 26259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:10.347784 26256 sys_catalog.cc:564] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:10.347810 26259 raft_consensus.cc:686] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [term 1 LEADER]: Becoming Leader. State: Replica: f94add5173a444399e36c75e764f9837, State: Running, Role: LEADER
I20260927 07:13:10.347895 26259 consensus_queue.cc:237] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94add5173a444399e36c75e764f9837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260927 07:13:10.348464 26261 sys_catalog.cc:455] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f94add5173a444399e36c75e764f9837. Latest consensus state: current_term: 1 leader_uuid: "f94add5173a444399e36c75e764f9837" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94add5173a444399e36c75e764f9837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } } }
I20260927 07:13:10.348493 26260 sys_catalog.cc:455] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f94add5173a444399e36c75e764f9837" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94add5173a444399e36c75e764f9837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } } }
I20260927 07:13:10.348553 26261 sys_catalog.cc:458] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:10.348573 26260 sys_catalog.cc:458] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:10.348740 26268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:10.349293 26268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:10.349730 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26188
I20260927 07:13:10.349826 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:10.350023 4793 external_mini_cluster.cc:1428] Setting key c1f86b6b6194d4ada6dad453f4a0715d
I20260927 07:13:10.350903 26268 catalog_manager.cc:1347] Generated new cluster ID: 3a84f6c710f54dbe8168c82740043799
I20260927 07:13:10.350952 26268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:10.352509 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:10.350435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:49903 (local address 127.0.0.1:35545)
0927 07:13:10.350621 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:10.350630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:10.351091 (+ 461us) server_negotiation.cc:407] Connection header received
0927 07:13:10.351221 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:10.351228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:10.351258 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:10.351411 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:10.352098 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:10.352102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:10.352102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:10.352107 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:10.352150 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:10.352165 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:10.352228 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:10.352293 (+ 65us) server_negotiation.cc:299] Negotiation successful
0927 07:13:10.352337 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260927 07:13:10.353554 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:10.354780 26268 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:10.354939 26268 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:10.360005 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:10.358844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:60845 (local address 127.0.0.1:35545)
0927 07:13:10.358877 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:10.358879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:10.359104 (+ 225us) server_negotiation.cc:407] Connection header received
0927 07:13:10.359129 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:10.359130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:10.359135 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:10.359179 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:10.359803 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:10.359805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:10.359805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:10.359810 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:10.359826 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:10.359830 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:10.359886 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:10.359925 (+ 39us) server_negotiation.cc:299] Negotiation successful
0927 07:13:10.359938 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 07:13:10.362965 26268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f94add5173a444399e36c75e764f9837: Generated new TSK 0
I20260927 07:13:10.363143 26268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:10.365053 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26188
2026-09-27T07:13:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1196 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:10 dist-test-slave-zw7p krb5kdc[26287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:10 dist-test-slave-zw7p krb5kdc[26287](info): set up 2 sockets
Sep 27 07:13:10 dist-test-slave-zw7p krb5kdc[26287](info): commencing operation
krb5kdc: starting...
W20260927 07:13:11.318502 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:11 dist-test-slave-zw7p krb5kdc[26287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:13:11.429153 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:37699
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:11.509164 26300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:11.509361 26300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:11.509385 26300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:11.510771 26300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:11.510821 26300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:11.510838 26300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:11.510852 26300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:11.510864 26300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:11.512313 26300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:37699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:11.512498 26300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:11.512665 26300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:11.517377 26306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:11.517370 26305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:11.517421 26300 server_base.cc:1034] running on GCE node
W20260927 07:13:11.517377 26308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:11.517798 26300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:11.518072 26300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:11.519214 26300 hybrid_clock.cc:648] HybridClock initialized: now 1790493191519201 us; error 26 us; skew 500 ppm
Sep 27 07:13:11 dist-test-slave-zw7p krb5kdc[26287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493191, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:11.521453 26300 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:13:11.521814 26300 webserver.cc:460] Webserver started at http://127.4.174.126:34959/ using document root <none> and password file <none>
I20260927 07:13:11.522006 26300 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:11.522044 26300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:11.522130 26300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:11.523138 26300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "a9d53861294b40d7b9dc3db1b40d6006"
format_stamp: "Formatted at 2026-09-27 07:13:11 on dist-test-slave-zw7p"
server_key: "bd56fce193128fdbe547740d9c7f2bc3"
server_key_iv: "8c07acdc63c810cf874bcad52abc5734"
server_key_version: "encryptionkey@0"
I20260927 07:13:11.523415 26300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "a9d53861294b40d7b9dc3db1b40d6006"
format_stamp: "Formatted at 2026-09-27 07:13:11 on dist-test-slave-zw7p"
server_key: "bd56fce193128fdbe547740d9c7f2bc3"
server_key_iv: "8c07acdc63c810cf874bcad52abc5734"
server_key_version: "encryptionkey@0"
I20260927 07:13:11.524571 26300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:13:11.525224 26315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:11.525410 26300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:11.525491 26300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "a9d53861294b40d7b9dc3db1b40d6006"
format_stamp: "Formatted at 2026-09-27 07:13:11 on dist-test-slave-zw7p"
server_key: "bd56fce193128fdbe547740d9c7f2bc3"
server_key_iv: "8c07acdc63c810cf874bcad52abc5734"
server_key_version: "encryptionkey@0"
I20260927 07:13:11.525558 26300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:11.545362 26300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:11.547308 26300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:11.547425 26300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:11.551046 26300 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:37699
I20260927 07:13:11.551116 26367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:37699 every 8 connection(s)
I20260927 07:13:11.551401 26300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:11.552274 26368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:11.554018 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26300
I20260927 07:13:11.554126 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:11.554296 4793 external_mini_cluster.cc:1428] Setting key 977cd6cbb938a5f1cf6d5e27b65501e9
I20260927 07:13:11.554737 26368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006: Bootstrap starting.
I20260927 07:13:11.555326 26368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:11.555606 26368 log.cc:824] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:11.556337 26368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006: No bootstrap required, opened a new log
I20260927 07:13:11.558337 26368 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d53861294b40d7b9dc3db1b40d6006" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37699 } }
I20260927 07:13:11.558470 26368 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:11.558501 26368 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9d53861294b40d7b9dc3db1b40d6006, State: Initialized, Role: FOLLOWER
I20260927 07:13:11.558646 26368 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d53861294b40d7b9dc3db1b40d6006" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37699 } }
I20260927 07:13:11.558720 26368 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:11.558750 26368 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:11.558811 26368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:11.559304 26368 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d53861294b40d7b9dc3db1b40d6006" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37699 } }
I20260927 07:13:11.559394 26368 leader_election.cc:302] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9d53861294b40d7b9dc3db1b40d6006; no voters:
I20260927 07:13:11.559543 26368 leader_election.cc:288] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:11.559576 26373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:11.559674 26373 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [term 1 LEADER]: Becoming Leader. State: Replica: a9d53861294b40d7b9dc3db1b40d6006, State: Running, Role: LEADER
I20260927 07:13:11.559791 26368 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:11.559803 26373 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d53861294b40d7b9dc3db1b40d6006" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37699 } }
Sep 27 07:13:11 dist-test-slave-zw7p krb5kdc[26287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:13:11.561045 26375 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9d53861294b40d7b9dc3db1b40d6006. Latest consensus state: current_term: 1 leader_uuid: "a9d53861294b40d7b9dc3db1b40d6006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d53861294b40d7b9dc3db1b40d6006" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37699 } } }
I20260927 07:13:11.561059 26374 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9d53861294b40d7b9dc3db1b40d6006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d53861294b40d7b9dc3db1b40d6006" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37699 } } }
I20260927 07:13:11.561233 26375 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:11.561247 26374 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006 [sys.catalog]: This master's current role is: LEADER
W20260927 07:13:11.561991 26388 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:11.562041 26388 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:13:11.562139 26390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:11.562285 26390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:11.563825 26390 catalog_manager.cc:1347] Generated new cluster ID: 40d7b1139edb445eaf4e33cda09aac4a
I20260927 07:13:11.563867 26390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:11.564862 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:11.554781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56757 (local address 127.4.174.126:37699)
0927 07:13:11.554924 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:11.554933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:11.554959 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:13:11.555365 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:11.555371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:11.555422 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:11.555556 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:11.556209 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:11.557113 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:11.558165 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:11.558609 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:11.558720 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:11.561251 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:11.561256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:11.561256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:11.561268 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:11.563128 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:11.563570 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:11.563573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:11.563573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:11.563675 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:11.563926 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:11.563927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:11.563927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:11.564238 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:11.564393 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:11.564553 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 07:13:11.564655 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
I20260927 07:13:11.565784 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:11.574029 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:11.567181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36935 (local address 127.4.174.126:37699)
0927 07:13:11.567214 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:11.567216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:11.567452 (+ 236us) server_negotiation.cc:407] Connection header received
0927 07:13:11.567497 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:11.567498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:11.567507 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:11.567569 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:11.568121 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:11.568871 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:11.569790 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:11.570113 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:11.570184 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:11.571478 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:11.571479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:11.571480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:11.571486 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:11.572931 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:11.573306 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:11.573307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:11.573307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:11.573362 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:11.573542 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:11.573543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:11.573543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:11.573667 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:11.573764 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:11.573864 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 07:13:11.573925 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:13:11.592495 26390 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:11.593119 26390 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:11.605019 26390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9d53861294b40d7b9dc3db1b40d6006: Generated new TSK 0
I20260927 07:13:11.605214 26390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:11.612594 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26300
W20260927 07:13:11.616460 26285 connection.cc:535] client connection to 127.4.174.126:37699 recv error: Network error: failed to read from TLS socket (remote: 127.4.174.126:37699): Connection reset by peer (error 104)
2026-09-27T07:13:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1246 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:11 dist-test-slave-zw7p krb5kdc[26399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:11 dist-test-slave-zw7p krb5kdc[26399](info): set up 2 sockets
Sep 27 07:13:11 dist-test-slave-zw7p krb5kdc[26399](info): commencing operation
krb5kdc: starting...
W20260927 07:13:12.550354 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:12 dist-test-slave-zw7p krb5kdc[26399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:13:12.661015 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:34225
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:34225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:12.740250 26412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:12.740453 26412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:12.740483 26412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:12.741600 26412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:12.741639 26412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:12.741658 26412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:12.741672 26412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:12.741688 26412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:12.743180 26412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:34225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:34225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:12.743425 26412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:12.743620 26412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:12.748157 26417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:12.748207 26418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:12.748217 26412 server_base.cc:1034] running on GCE node
W20260927 07:13:12.748157 26420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:12.748584 26412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:12.748845 26412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:12.749981 26412 hybrid_clock.cc:648] HybridClock initialized: now 1790493192749966 us; error 27 us; skew 500 ppm
Sep 27 07:13:12 dist-test-slave-zw7p krb5kdc[26399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493192, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:12.755980 26412 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:13:12.756369 26412 webserver.cc:460] Webserver started at http://127.4.174.126:42365/ using document root <none> and password file <none>
I20260927 07:13:12.756567 26412 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:12.756613 26412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:12.756729 26412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:12.757736 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "bb35b3769d274c67b881ed7d263cc4be"
format_stamp: "Formatted at 2026-09-27 07:13:12 on dist-test-slave-zw7p"
server_key: "a34ff940e074ee7d3d7ee598d7705abf"
server_key_iv: "380b9bf2d3f70e0437ba6ffcf4f6eabe"
server_key_version: "encryptionkey@0"
I20260927 07:13:12.758047 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "bb35b3769d274c67b881ed7d263cc4be"
format_stamp: "Formatted at 2026-09-27 07:13:12 on dist-test-slave-zw7p"
server_key: "a34ff940e074ee7d3d7ee598d7705abf"
server_key_iv: "380b9bf2d3f70e0437ba6ffcf4f6eabe"
server_key_version: "encryptionkey@0"
I20260927 07:13:12.759270 26412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:13:12.759927 26427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:12.760079 26412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:12.760155 26412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "bb35b3769d274c67b881ed7d263cc4be"
format_stamp: "Formatted at 2026-09-27 07:13:12 on dist-test-slave-zw7p"
server_key: "a34ff940e074ee7d3d7ee598d7705abf"
server_key_iv: "380b9bf2d3f70e0437ba6ffcf4f6eabe"
server_key_version: "encryptionkey@0"
I20260927 07:13:12.760212 26412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:12.790367 26412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:12.792150 26412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:12.792284 26412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:12.796049 26412 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:34225
I20260927 07:13:12.796097 26479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:34225 every 8 connection(s)
I20260927 07:13:12.796427 26412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:12.797298 26480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:12.799551 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be: Bootstrap starting.
I20260927 07:13:12.800094 26480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:12.800344 26480 log.cc:824] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:12.801010 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be: No bootstrap required, opened a new log
I20260927 07:13:12.802592 26480 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb35b3769d274c67b881ed7d263cc4be" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34225 } }
I20260927 07:13:12.802742 26480 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:12.802772 26480 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb35b3769d274c67b881ed7d263cc4be, State: Initialized, Role: FOLLOWER
I20260927 07:13:12.802928 26480 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb35b3769d274c67b881ed7d263cc4be" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34225 } }
I20260927 07:13:12.803005 26480 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:12.803043 26480 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:12.803092 26480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:12.803879 26480 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb35b3769d274c67b881ed7d263cc4be" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34225 } }
I20260927 07:13:12.803977 26480 leader_election.cc:302] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb35b3769d274c67b881ed7d263cc4be; no voters:
I20260927 07:13:12.804126 26480 leader_election.cc:288] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:12.804162 26483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:12.804296 26483 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [term 1 LEADER]: Becoming Leader. State: Replica: bb35b3769d274c67b881ed7d263cc4be, State: Running, Role: LEADER
I20260927 07:13:12.804358 26480 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:12.804409 26483 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb35b3769d274c67b881ed7d263cc4be" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34225 } }
I20260927 07:13:12.804777 26484 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb35b3769d274c67b881ed7d263cc4be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb35b3769d274c67b881ed7d263cc4be" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34225 } } }
I20260927 07:13:12.804780 26485 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb35b3769d274c67b881ed7d263cc4be. Latest consensus state: current_term: 1 leader_uuid: "bb35b3769d274c67b881ed7d263cc4be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb35b3769d274c67b881ed7d263cc4be" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34225 } } }
I20260927 07:13:12.804937 26484 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:12.804943 26485 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:12.805424 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26412
I20260927 07:13:12.805445 26492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:12.805537 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:12.805716 4793 external_mini_cluster.cc:1428] Setting key 8965d36aca5ec4571754cfb2fd5a7095
I20260927 07:13:12.805943 26492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:12.807438 26492 catalog_manager.cc:1347] Generated new cluster ID: 5b10ba330cc14dcfb502ee894556e7c2
I20260927 07:13:12.807488 26492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:12.810672 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:12.806162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34641 (local address 127.4.174.126:34225)
0927 07:13:12.806507 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:12.806517 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:12.806547 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:13:12.806684 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:12.806691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:12.806765 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:12.806895 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:12.807209 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:12.808180 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:12.808905 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:12.809329 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:12.809420 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:12.810055 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:12.810059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:12.810060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:12.810219 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:12.810248 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:12.810258 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:12.810323 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:12.810412 (+ 89us) server_negotiation.cc:299] Negotiation successful
0927 07:13:12.810438 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":59,"threads_started":1}
I20260927 07:13:12.811507 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:12.815485 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:12.812722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48149 (local address 127.4.174.126:34225)
0927 07:13:12.812753 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:12.812755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:12.812849 (+ 94us) server_negotiation.cc:407] Connection header received
0927 07:13:12.812902 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:12.812903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:12.812913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:12.812957 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:12.813126 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:12.813650 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:12.814340 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:12.814582 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:12.814635 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:12.815270 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:12.815271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:12.815272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:12.815276 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:12.815290 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:12.815294 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:12.815337 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:12.815403 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 07:13:12.815415 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:13:12.822723 26492 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:12.823247 26492 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:12.830642 26492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb35b3769d274c67b881ed7d263cc4be: Generated new TSK 0
I20260927 07:13:12.830828 26492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:12.833796 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26412
W20260927 07:13:12.837649 26397 connection.cc:535] client connection to 127.4.174.126:34225 recv error: Network error: failed to read from TLS socket (remote: 127.4.174.126:34225): Connection reset by peer (error 104)
2026-09-27T07:13:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1221 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:12 dist-test-slave-zw7p krb5kdc[26511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:12 dist-test-slave-zw7p krb5kdc[26511](info): set up 2 sockets
Sep 27 07:13:12 dist-test-slave-zw7p krb5kdc[26511](info): commencing operation
krb5kdc: starting...
W20260927 07:13:13.778234 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:13 dist-test-slave-zw7p krb5kdc[26511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:13:13.889599 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:45949
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:45949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:13.968593 26524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:13.968809 26524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:13.968842 26524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:13.969981 26524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:13.970031 26524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:13.970052 26524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:13.970067 26524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:13.970082 26524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:13.971635 26524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:45949
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:45949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:13.971887 26524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:13.972098 26524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:13.976778 26529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:13.976771 26532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:13.976778 26530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:13.976863 26524 server_base.cc:1034] running on GCE node
I20260927 07:13:13.977177 26524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:13.977456 26524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:13.978611 26524 hybrid_clock.cc:648] HybridClock initialized: now 1790493193978594 us; error 28 us; skew 500 ppm
Sep 27 07:13:13 dist-test-slave-zw7p krb5kdc[26511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493193, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:13.980928 26524 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:13:13.981335 26524 webserver.cc:460] Webserver started at http://127.4.174.126:40109/ using document root <none> and password file <none>
I20260927 07:13:13.981534 26524 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:13.981575 26524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:13.981667 26524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:13.982700 26524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "7d8194cd947a468fb0b9379bd49f1ed3"
format_stamp: "Formatted at 2026-09-27 07:13:13 on dist-test-slave-zw7p"
server_key: "e186eadf45b01b593f77b051d7552933"
server_key_iv: "2de6432325616f8cd4e5fe641eb78826"
server_key_version: "encryptionkey@0"
I20260927 07:13:13.982991 26524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "7d8194cd947a468fb0b9379bd49f1ed3"
format_stamp: "Formatted at 2026-09-27 07:13:13 on dist-test-slave-zw7p"
server_key: "e186eadf45b01b593f77b051d7552933"
server_key_iv: "2de6432325616f8cd4e5fe641eb78826"
server_key_version: "encryptionkey@0"
I20260927 07:13:13.984135 26524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 07:13:13.984730 26539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:13.984871 26524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 07:13:13.984943 26524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "7d8194cd947a468fb0b9379bd49f1ed3"
format_stamp: "Formatted at 2026-09-27 07:13:13 on dist-test-slave-zw7p"
server_key: "e186eadf45b01b593f77b051d7552933"
server_key_iv: "2de6432325616f8cd4e5fe641eb78826"
server_key_version: "encryptionkey@0"
I20260927 07:13:13.985008 26524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:13.988148 26524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:13.990166 26524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:13.990288 26524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:13.993909 26524 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:45949
I20260927 07:13:13.993937 26591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:45949 every 8 connection(s)
I20260927 07:13:13.994234 26524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:13.995229 26592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:13.997402 26592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3: Bootstrap starting.
I20260927 07:13:13.997941 26592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:13.998168 26592 log.cc:824] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:13.998823 26592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3: No bootstrap required, opened a new log
I20260927 07:13:14.000340 26592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d8194cd947a468fb0b9379bd49f1ed3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45949 } }
I20260927 07:13:14.000475 26592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:14.000495 26592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d8194cd947a468fb0b9379bd49f1ed3, State: Initialized, Role: FOLLOWER
I20260927 07:13:14.000607 26592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d8194cd947a468fb0b9379bd49f1ed3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45949 } }
I20260927 07:13:14.000655 26592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:14.000679 26592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:14.000708 26592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:14.001426 26592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d8194cd947a468fb0b9379bd49f1ed3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45949 } }
I20260927 07:13:14.001507 26592 leader_election.cc:302] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d8194cd947a468fb0b9379bd49f1ed3; no voters:
I20260927 07:13:14.001631 26592 leader_election.cc:288] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:14.001670 26595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:14.001792 26595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [term 1 LEADER]: Becoming Leader. State: Replica: 7d8194cd947a468fb0b9379bd49f1ed3, State: Running, Role: LEADER
I20260927 07:13:14.001860 26592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:14.001879 26595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d8194cd947a468fb0b9379bd49f1ed3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45949 } }
I20260927 07:13:14.002213 26597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d8194cd947a468fb0b9379bd49f1ed3. Latest consensus state: current_term: 1 leader_uuid: "7d8194cd947a468fb0b9379bd49f1ed3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d8194cd947a468fb0b9379bd49f1ed3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45949 } } }
I20260927 07:13:14.002206 26596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d8194cd947a468fb0b9379bd49f1ed3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d8194cd947a468fb0b9379bd49f1ed3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 45949 } } }
I20260927 07:13:14.002297 26597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:14.002303 26596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:14.002522 26602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:14.003067 26602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:14.003767 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26524
I20260927 07:13:14.003860 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:14.004014 4793 external_mini_cluster.cc:1428] Setting key cbacc0f56f9a3173155d9a7bfd7f0319
I20260927 07:13:14.004535 26602 catalog_manager.cc:1347] Generated new cluster ID: 9c775907569c4125a3962cd8f53df0e9
I20260927 07:13:14.004580 26602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:14.006121 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:14.004426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55821 (local address 127.4.174.126:45949)
0927 07:13:14.004735 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:14.004744 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:14.004774 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:13:14.004877 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:14.004884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:14.004912 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:14.005028 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:14.005641 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:14.005644 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:14.005645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:14.005649 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:14.005687 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:14.005701 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:14.005755 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:14.005833 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 07:13:14.005949 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":65,"threads_started":1}
I20260927 07:13:14.007282 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:14.009755 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:14.008723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41265 (local address 127.4.174.126:45949)
0927 07:13:14.008757 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:14.008759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:14.008900 (+ 141us) server_negotiation.cc:407] Connection header received
0927 07:13:14.008969 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:14.008970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:14.008976 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:14.009032 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:14.009554 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:14.009555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:14.009555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:14.009559 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:14.009573 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:14.009577 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:14.009630 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:14.009677 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 07:13:14.009690 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:13:14.018662 26602 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:14.018854 26602 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:14.029475 26602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d8194cd947a468fb0b9379bd49f1ed3: Generated new TSK 0
I20260927 07:13:14.029700 26602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:14.036911 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26524
2026-09-27T07:13:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1202 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:14 dist-test-slave-zw7p krb5kdc[26623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:14 dist-test-slave-zw7p krb5kdc[26623](info): set up 2 sockets
Sep 27 07:13:14 dist-test-slave-zw7p krb5kdc[26623](info): commencing operation
krb5kdc: starting...
W20260927 07:13:14.990345 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:15 dist-test-slave-zw7p krb5kdc[26623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:13:15.101492 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35889
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:42653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:35889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:15.179461 26636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:15.179662 26636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:15.179693 26636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:15.180817 26636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:15.180857 26636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:15.180874 26636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:15.180889 26636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:15.180904 26636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:15.182343 26636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:42653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:35889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:35889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:15.182603 26636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:15.182808 26636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:15.187314 26642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:15.187314 26644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:15.187316 26641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:15.187590 26636 server_base.cc:1034] running on GCE node
I20260927 07:13:15.187834 26636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:15.188105 26636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:15.189252 26636 hybrid_clock.cc:648] HybridClock initialized: now 1790493195189237 us; error 28 us; skew 500 ppm
Sep 27 07:13:15 dist-test-slave-zw7p krb5kdc[26623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493195, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:15.191655 26636 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:13:15.192082 26636 webserver.cc:460] Webserver started at http://127.4.174.126:35183/ using document root <none> and password file <none>
I20260927 07:13:15.192293 26636 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:15.192337 26636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:15.192458 26636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:15.193464 26636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "7dd96244a93f4bb69681ecb4a8322aa0"
format_stamp: "Formatted at 2026-09-27 07:13:15 on dist-test-slave-zw7p"
server_key: "222670bddc4fae0ce0025f7435d26521"
server_key_iv: "e56101dcf52dc0f0821b760f2c944f6a"
server_key_version: "encryptionkey@0"
I20260927 07:13:15.193773 26636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "7dd96244a93f4bb69681ecb4a8322aa0"
format_stamp: "Formatted at 2026-09-27 07:13:15 on dist-test-slave-zw7p"
server_key: "222670bddc4fae0ce0025f7435d26521"
server_key_iv: "e56101dcf52dc0f0821b760f2c944f6a"
server_key_version: "encryptionkey@0"
I20260927 07:13:15.194993 26636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:13:15.195608 26651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:15.195739 26636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:15.195815 26636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "7dd96244a93f4bb69681ecb4a8322aa0"
format_stamp: "Formatted at 2026-09-27 07:13:15 on dist-test-slave-zw7p"
server_key: "222670bddc4fae0ce0025f7435d26521"
server_key_iv: "e56101dcf52dc0f0821b760f2c944f6a"
server_key_version: "encryptionkey@0"
I20260927 07:13:15.195875 26636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:15.230033 26636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:15.231825 26636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:15.231963 26636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:15.235560 26636 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:35889
I20260927 07:13:15.235608 26703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:35889 every 8 connection(s)
I20260927 07:13:15.235966 26636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:15.236769 26704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:15.238967 26704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0: Bootstrap starting.
I20260927 07:13:15.239461 26704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:15.239684 26704 log.cc:824] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:15.240263 26704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0: No bootstrap required, opened a new log
I20260927 07:13:15.241766 26704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd96244a93f4bb69681ecb4a8322aa0" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35889 } }
I20260927 07:13:15.241899 26704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:15.241919 26704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd96244a93f4bb69681ecb4a8322aa0, State: Initialized, Role: FOLLOWER
I20260927 07:13:15.242024 26704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd96244a93f4bb69681ecb4a8322aa0" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35889 } }
I20260927 07:13:15.242100 26704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:15.242132 26704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:15.242164 26704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:15.242955 26704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd96244a93f4bb69681ecb4a8322aa0" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35889 } }
I20260927 07:13:15.243041 26704 leader_election.cc:302] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dd96244a93f4bb69681ecb4a8322aa0; no voters:
I20260927 07:13:15.243155 26704 leader_election.cc:288] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:15.243197 26707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:15.243367 26704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:15.243372 26707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd96244a93f4bb69681ecb4a8322aa0, State: Running, Role: LEADER
I20260927 07:13:15.243484 26707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd96244a93f4bb69681ecb4a8322aa0" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35889 } }
I20260927 07:13:15.243834 26708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dd96244a93f4bb69681ecb4a8322aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd96244a93f4bb69681ecb4a8322aa0" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35889 } } }
I20260927 07:13:15.243855 26709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dd96244a93f4bb69681ecb4a8322aa0. Latest consensus state: current_term: 1 leader_uuid: "7dd96244a93f4bb69681ecb4a8322aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd96244a93f4bb69681ecb4a8322aa0" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 35889 } } }
I20260927 07:13:15.243923 26708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:15.243932 26709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:15.244132 26716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:15.244622 26716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:15.246133 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26636
I20260927 07:13:15.246215 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:15.246366 4793 external_mini_cluster.cc:1428] Setting key 080c5a97f6658426ca28755e1ff84f0b
I20260927 07:13:15.246533 26716 catalog_manager.cc:1347] Generated new cluster ID: 73a7939e8c674eaa8f160fd27f2a954e
I20260927 07:13:15.246592 26716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 07:13:15 dist-test-slave-zw7p krb5kdc[26623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:13:15.259238 26716 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:15.259817 26716 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:15.260993 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:15.246785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56287 (local address 127.4.174.126:35889)
0927 07:13:15.250667 (+ 3882us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:15.250676 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:15.250709 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:13:15.250821 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:15.250828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:15.250887 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:15.251013 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:15.251718 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:15.253057 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:15.254126 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:15.254752 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:15.254843 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:15.257552 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:15.257556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:15.257556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:15.257569 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:15.259454 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:15.259805 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:15.259808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:15.259809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:15.259943 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:15.260126 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:15.260127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:15.260127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:15.260396 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:15.260544 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:15.260615 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 07:13:15.260757 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3832,"thread_start_us":66,"threads_started":1}
I20260927 07:13:15.261794 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:15.270004 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:15.263159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37341 (local address 127.4.174.126:35889)
0927 07:13:15.263192 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:15.263194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:15.263247 (+ 53us) server_negotiation.cc:407] Connection header received
0927 07:13:15.263312 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:15.263312 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:15.263321 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:15.263370 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:15.263915 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:15.264696 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:15.265957 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:15.266292 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:15.266351 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:15.267558 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:15.267559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:15.267560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:15.267567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:15.268984 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:15.269319 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:15.269320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:15.269320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:15.269374 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:15.269530 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:15.269530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:15.269531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:15.269648 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:15.269759 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:15.269822 (+ 63us) server_negotiation.cc:299] Negotiation successful
0927 07:13:15.269903 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:13:15.271100 26716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dd96244a93f4bb69681ecb4a8322aa0: Generated new TSK 0
I20260927 07:13:15.271267 26716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:15.276422 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26636
2026-09-27T07:13:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1239 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:15 dist-test-slave-zw7p krb5kdc[26735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:15 dist-test-slave-zw7p krb5kdc[26735](info): set up 2 sockets
Sep 27 07:13:15 dist-test-slave-zw7p krb5kdc[26735](info): commencing operation
krb5kdc: starting...
W20260927 07:13:16.218411 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:16 dist-test-slave-zw7p krb5kdc[26735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:13:16.329377 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:34859
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:36003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:34859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:16.407419 26748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:16.407630 26748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:16.407653 26748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:16.408787 26748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:16.408826 26748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:16.408838 26748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:16.408849 26748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:16.408860 26748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:16.410290 26748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:36003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:34859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:34859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:16.410494 26748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:16.410694 26748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:16.415283 26756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:16.415310 26753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:16.415292 26754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:16.415498 26748 server_base.cc:1034] running on GCE node
I20260927 07:13:16.415735 26748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:16.416000 26748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:16.417135 26748 hybrid_clock.cc:648] HybridClock initialized: now 1790493196417123 us; error 27 us; skew 500 ppm
Sep 27 07:13:16 dist-test-slave-zw7p krb5kdc[26735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493196, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:16.419441 26748 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:13:16.419886 26748 webserver.cc:460] Webserver started at http://127.4.174.126:44831/ using document root <none> and password file <none>
I20260927 07:13:16.420097 26748 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:16.420145 26748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:16.420266 26748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:16.421310 26748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "d78d033ce1524cfba8718c331bc2652e"
format_stamp: "Formatted at 2026-09-27 07:13:16 on dist-test-slave-zw7p"
server_key: "10ad28bcf3d09105994b04888958de60"
server_key_iv: "637ea7397bd82959837a071bf2ec0502"
server_key_version: "encryptionkey@0"
I20260927 07:13:16.421643 26748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "d78d033ce1524cfba8718c331bc2652e"
format_stamp: "Formatted at 2026-09-27 07:13:16 on dist-test-slave-zw7p"
server_key: "10ad28bcf3d09105994b04888958de60"
server_key_iv: "637ea7397bd82959837a071bf2ec0502"
server_key_version: "encryptionkey@0"
I20260927 07:13:16.422978 26748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:13:16.423635 26763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:16.423786 26748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:16.423877 26748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "d78d033ce1524cfba8718c331bc2652e"
format_stamp: "Formatted at 2026-09-27 07:13:16 on dist-test-slave-zw7p"
server_key: "10ad28bcf3d09105994b04888958de60"
server_key_iv: "637ea7397bd82959837a071bf2ec0502"
server_key_version: "encryptionkey@0"
I20260927 07:13:16.423936 26748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:16.435082 26748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:16.436800 26748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:16.436934 26748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:16.440811 26748 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:34859
I20260927 07:13:16.440855 26815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:34859 every 8 connection(s)
I20260927 07:13:16.441181 26748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:16.442013 26816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:16.443698 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26748
I20260927 07:13:16.443801 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:16.443991 4793 external_mini_cluster.cc:1428] Setting key 3a870296d9fabb2fb3612ea2a372f44a
I20260927 07:13:16.444679 26816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e: Bootstrap starting.
I20260927 07:13:16.445266 26816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:16.445530 26816 log.cc:824] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:16.446178 26816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e: No bootstrap required, opened a new log
I20260927 07:13:16.448002 26816 raft_consensus.cc:348] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d033ce1524cfba8718c331bc2652e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34859 } }
I20260927 07:13:16.448154 26816 raft_consensus.cc:374] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:16.448189 26816 raft_consensus.cc:729] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78d033ce1524cfba8718c331bc2652e, State: Initialized, Role: FOLLOWER
I20260927 07:13:16.448315 26816 consensus_queue.cc:260] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d033ce1524cfba8718c331bc2652e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34859 } }
I20260927 07:13:16.448397 26816 raft_consensus.cc:388] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:16.448486 26816 raft_consensus.cc:482] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:16.448539 26816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:16.449108 26816 raft_consensus.cc:504] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d033ce1524cfba8718c331bc2652e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34859 } }
I20260927 07:13:16.449157 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:16.444539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43863 (local address 127.4.174.126:34859)
0927 07:13:16.444718 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:16.444726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:16.444759 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:13:16.445236 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:16.445244 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:16.445299 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:16.445457 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:16.445677 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:16.446636 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:16.447381 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:16.447794 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:16.447909 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:16.448578 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:16.448582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:16.448582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:16.448756 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:16.448789 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:16.448800 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:16.448851 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:16.448929 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 07:13:16.448994 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260927 07:13:16.449204 26816 leader_election.cc:302] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d78d033ce1524cfba8718c331bc2652e; no voters:
I20260927 07:13:16.449363 26816 leader_election.cc:288] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:16.449414 26821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:16.449541 26821 raft_consensus.cc:686] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [term 1 LEADER]: Becoming Leader. State: Replica: d78d033ce1524cfba8718c331bc2652e, State: Running, Role: LEADER
I20260927 07:13:16.449623 26816 sys_catalog.cc:564] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:16.449628 26821 consensus_queue.cc:237] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d033ce1524cfba8718c331bc2652e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34859 } }
I20260927 07:13:16.450079 26823 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d78d033ce1524cfba8718c331bc2652e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d033ce1524cfba8718c331bc2652e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34859 } } }
I20260927 07:13:16.450153 26823 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:16.450217 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:16.450362 26822 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d78d033ce1524cfba8718c331bc2652e. Latest consensus state: current_term: 1 leader_uuid: "d78d033ce1524cfba8718c331bc2652e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d033ce1524cfba8718c331bc2652e" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 34859 } } }
I20260927 07:13:16.450474 26822 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:16.450700 26829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:16.451195 26829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:16.452667 26829 catalog_manager.cc:1347] Generated new cluster ID: 2d01ea8d247c4c1a9bff295320992674
I20260927 07:13:16.452710 26829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:16.458384 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:16.454882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41611 (local address 127.4.174.126:34859)
0927 07:13:16.454916 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:16.454918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:16.455088 (+ 170us) server_negotiation.cc:407] Connection header received
0927 07:13:16.455156 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:16.455157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:16.455166 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:16.455227 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:16.455372 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:16.456108 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:16.457136 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:16.457421 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:16.457472 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:16.458162 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:16.458163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:16.458164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:16.458168 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:16.458183 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:16.458188 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:16.458228 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:16.458282 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 07:13:16.458308 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:13:16.468228 26829 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:16.468753 26829 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:16.472771 26829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d78d033ce1524cfba8718c331bc2652e: Generated new TSK 0
I20260927 07:13:16.472954 26829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:16.482692 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26748
2026-09-27T07:13:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1206 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:16 dist-test-slave-zw7p krb5kdc[26847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:16 dist-test-slave-zw7p krb5kdc[26847](info): set up 2 sockets
Sep 27 07:13:16 dist-test-slave-zw7p krb5kdc[26847](info): commencing operation
krb5kdc: starting...
W20260927 07:13:17.422354 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:17 dist-test-slave-zw7p krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:13:17.533099 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:36507
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:36507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:17.612402 26860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:17.612607 26860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:17.612635 26860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:17.613762 26860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:17.613812 26860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:17.613832 26860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:17.613848 26860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:17.613871 26860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:17.615361 26860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:36507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:36507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:17.615613 26860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:17.615818 26860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:17.620466 26865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:17.620476 26866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:17.621395 26868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:17.621539 26860 server_base.cc:1034] running on GCE node
I20260927 07:13:17.621776 26860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:17.622051 26860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:17.623193 26860 hybrid_clock.cc:648] HybridClock initialized: now 1790493197623176 us; error 27 us; skew 500 ppm
Sep 27 07:13:17 dist-test-slave-zw7p krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493197, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:17.625478 26860 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:13:17.625919 26860 webserver.cc:460] Webserver started at http://127.4.174.126:37679/ using document root <none> and password file <none>
I20260927 07:13:17.626122 26860 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:17.626170 26860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:17.626291 26860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:17.627350 26860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "a230a3e5bc7445fb887e5b7801a95cc3"
format_stamp: "Formatted at 2026-09-27 07:13:17 on dist-test-slave-zw7p"
server_key: "44c62bde2a83f3ef776ec9e66d69f369"
server_key_iv: "5839da1d1adeedc824c004f9ba2ba6c5"
server_key_version: "encryptionkey@0"
I20260927 07:13:17.627664 26860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "a230a3e5bc7445fb887e5b7801a95cc3"
format_stamp: "Formatted at 2026-09-27 07:13:17 on dist-test-slave-zw7p"
server_key: "44c62bde2a83f3ef776ec9e66d69f369"
server_key_iv: "5839da1d1adeedc824c004f9ba2ba6c5"
server_key_version: "encryptionkey@0"
I20260927 07:13:17.628852 26860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:13:17.629438 26875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:17.629585 26860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:17.629662 26860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "a230a3e5bc7445fb887e5b7801a95cc3"
format_stamp: "Formatted at 2026-09-27 07:13:17 on dist-test-slave-zw7p"
server_key: "44c62bde2a83f3ef776ec9e66d69f369"
server_key_iv: "5839da1d1adeedc824c004f9ba2ba6c5"
server_key_version: "encryptionkey@0"
I20260927 07:13:17.629719 26860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:17.633023 26860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:17.635077 26860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:17.635205 26860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:17.638949 26860 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:36507
I20260927 07:13:17.638985 26927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:36507 every 8 connection(s)
I20260927 07:13:17.639283 26860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:17.640187 26928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:17.642309 26928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3: Bootstrap starting.
I20260927 07:13:17.642841 26928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:17.643071 26928 log.cc:824] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:17.643678 26928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3: No bootstrap required, opened a new log
I20260927 07:13:17.645181 26928 raft_consensus.cc:348] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a230a3e5bc7445fb887e5b7801a95cc3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 36507 } }
I20260927 07:13:17.645313 26928 raft_consensus.cc:374] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:13:17.645334 26928 raft_consensus.cc:729] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a230a3e5bc7445fb887e5b7801a95cc3, State: Initialized, Role: FOLLOWER
I20260927 07:13:17.645432 26928 consensus_queue.cc:260] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a230a3e5bc7445fb887e5b7801a95cc3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 36507 } }
I20260927 07:13:17.645476 26928 raft_consensus.cc:388] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:13:17.645499 26928 raft_consensus.cc:482] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:13:17.645529 26928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:17.646274 26928 raft_consensus.cc:504] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a230a3e5bc7445fb887e5b7801a95cc3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 36507 } }
I20260927 07:13:17.646356 26928 leader_election.cc:302] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a230a3e5bc7445fb887e5b7801a95cc3; no voters:
I20260927 07:13:17.646472 26928 leader_election.cc:288] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:13:17.646520 26931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:17.646706 26928 sys_catalog.cc:564] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:17.646713 26931 raft_consensus.cc:686] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [term 1 LEADER]: Becoming Leader. State: Replica: a230a3e5bc7445fb887e5b7801a95cc3, State: Running, Role: LEADER
I20260927 07:13:17.646867 26931 consensus_queue.cc:237] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a230a3e5bc7445fb887e5b7801a95cc3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 36507 } }
I20260927 07:13:17.647243 26932 sys_catalog.cc:455] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a230a3e5bc7445fb887e5b7801a95cc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a230a3e5bc7445fb887e5b7801a95cc3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 36507 } } }
I20260927 07:13:17.647325 26932 sys_catalog.cc:458] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:17.647383 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26860
I20260927 07:13:17.647478 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:17.647655 4793 external_mini_cluster.cc:1428] Setting key 6eec01f400a9d9c55d44e3cc4743d943
I20260927 07:13:17.647773 26933 sys_catalog.cc:455] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a230a3e5bc7445fb887e5b7801a95cc3. Latest consensus state: current_term: 1 leader_uuid: "a230a3e5bc7445fb887e5b7801a95cc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a230a3e5bc7445fb887e5b7801a95cc3" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 36507 } } }
I20260927 07:13:17.647857 26933 sys_catalog.cc:458] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:17.647969 26940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:17.648794 26940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:17.649740 26949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:17.648247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52477 (local address 127.4.174.126:36507)
0927 07:13:17.648386 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:17.648396 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:17.648431 (+ 35us) server_negotiation.cc:407] Connection header received
0927 07:13:17.648533 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:17.648538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:17.648561 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:17.648684 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:17.649335 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:17.649338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:17.649339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:17.649343 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:17.649382 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:17.649397 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:17.649459 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:17.649535 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 07:13:17.649575 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
I20260927 07:13:17.650453 26940 catalog_manager.cc:1347] Generated new cluster ID: 39e61155b680409ebb6e576f7a561f9a
I20260927 07:13:17.650498 26940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:17.650672 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:17.654201 26940 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:17.654336 26940 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:17.655679 26949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:17.654501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47227 (local address 127.4.174.126:36507)
0927 07:13:17.654535 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:17.654536 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:17.654721 (+ 185us) server_negotiation.cc:407] Connection header received
0927 07:13:17.654815 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:17.654816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:17.654820 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:17.654843 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:17.655452 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:17.655453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:17.655453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:13:17.655456 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:17.655469 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:13:17.655473 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:17.655499 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:17.655598 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 07:13:17.655610 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 07:13:17.662209 26940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a230a3e5bc7445fb887e5b7801a95cc3: Generated new TSK 0
I20260927 07:13:17.662382 26940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:17.665869 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26860
2026-09-27T07:13:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1182 ms)
[----------] 18 tests from AuthTokenIssuingTest (21893 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:17 dist-test-slave-zw7p krb5kdc[26959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:17 dist-test-slave-zw7p krb5kdc[26959](info): set up 2 sockets
Sep 27 07:13:17 dist-test-slave-zw7p krb5kdc[26959](info): commencing operation
krb5kdc: starting...
W20260927 07:13:18.606465 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:18 dist-test-slave-zw7p krb5kdc[26959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:13:18.721367 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:37187
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:37741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:37187,127.4.174.125:34927,127.4.174.124:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:18.801127 26972 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:13:18.801344 26972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:18.801376 26972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:18.801403 26972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:18.802526 26972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:18.802585 26972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:18.802605 26972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:18.802623 26972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:18.802645 26972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:18.804124 26972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:37741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:37187,127.4.174.125:34927,127.4.174.124:37545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:37187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:18.804375 26972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:18.804581 26972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:18.809158 26980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:18.809238 26978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:18.809167 26977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:18.809394 26972 server_base.cc:1034] running on GCE node
I20260927 07:13:18.809608 26972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:18.809870 26972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:18.811030 26972 hybrid_clock.cc:648] HybridClock initialized: now 1790493198811015 us; error 27 us; skew 500 ppm
Sep 27 07:13:18 dist-test-slave-zw7p krb5kdc[26959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:18.816247 26972 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:13:18.816717 26972 webserver.cc:460] Webserver started at http://127.4.174.126:40747/ using document root <none> and password file <none>
I20260927 07:13:18.816941 26972 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:18.816993 26972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:18.817104 26972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:18.818182 26972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "44f09986f4d44f40b014698c1bd69c05"
format_stamp: "Formatted at 2026-09-27 07:13:18 on dist-test-slave-zw7p"
server_key: "d7b66f81a2445e72e3b30bbd179906ae"
server_key_iv: "efcd85a6b9020d49beb312234a9fab01"
server_key_version: "encryptionkey@0"
I20260927 07:13:18.818526 26972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "44f09986f4d44f40b014698c1bd69c05"
format_stamp: "Formatted at 2026-09-27 07:13:18 on dist-test-slave-zw7p"
server_key: "d7b66f81a2445e72e3b30bbd179906ae"
server_key_iv: "efcd85a6b9020d49beb312234a9fab01"
server_key_version: "encryptionkey@0"
I20260927 07:13:18.819821 26972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:13:18.820501 26987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:18.820638 26972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:18.820721 26972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "44f09986f4d44f40b014698c1bd69c05"
format_stamp: "Formatted at 2026-09-27 07:13:18 on dist-test-slave-zw7p"
server_key: "d7b66f81a2445e72e3b30bbd179906ae"
server_key_iv: "efcd85a6b9020d49beb312234a9fab01"
server_key_version: "encryptionkey@0"
I20260927 07:13:18.820781 26972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:18.830245 26972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:18.832331 26972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:18.832465 26972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:18.836167 26972 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:37187
I20260927 07:13:18.836212 27039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:37187 every 8 connection(s)
I20260927 07:13:18.836544 26972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:18.837507 27040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:18.839126 27040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:13:18 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:13:18.846022 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 26972
I20260927 07:13:18.846124 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
I20260927 07:13:18.846282 4793 external_mini_cluster.cc:1428] Setting key fd9c45ab886e7458c99921973db32c84
I20260927 07:13:18.848552 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.839811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:37187 (local address 127.0.0.1:49734)
0927 07:13:18.839973 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:18.840010 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:18.840476 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:18.840726 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:18.840729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:18.841008 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:18.841156 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:18.841161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:18.842155 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:18.842156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:18.842869 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:18.842872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:18.843326 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:18.843327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:18.843387 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:18.843825 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:18.843834 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:18.845209 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:18.846921 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:18.846929 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:18.846932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:18.847202 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:18.847441 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:18.847442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:18.847442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:18.847560 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:18.848009 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:18.848016 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:18.848080 (+ 64us) client_negotiation.cc:769] Sending connection context
0927 07:13:18.848165 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:13:18.848328 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260927 07:13:18.848558 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.839983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.4.174.126:37187)
0927 07:13:18.840101 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:18.840106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:18.840137 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:13:18.840625 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:18.840627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:18.840643 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:18.840714 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:18.841246 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:18.842068 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:18.842949 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:18.843242 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:18.843334 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:18.845285 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:18.845292 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:18.845292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:18.845299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:18.846838 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:18.847305 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:18.847307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:18.847307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:18.847385 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:18.847630 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:18.847631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:18.847631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:18.847935 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:18.848012 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:18.848207 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 07:13:18.848398 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
I20260927 07:13:18.849298 27040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:18.849625 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.849490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59146)
0927 07:13:18.849542 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:18.849599 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 07:13:18.849763 26988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
I20260927 07:13:18.850179 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.850075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59152)
0927 07:13:18.850124 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:18.850156 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 07:13:18.850291 27040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:34927: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.174.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab.
Entry for principal kudu/127.4.174.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab.
I20260927 07:13:18.869503 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.125:34927
--webserver_interface=127.4.174.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab
--principal=kudu/127.4.174.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:37741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:37187,127.4.174.125:34927,127.4.174.124:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:13:18.899529 27040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } attempt: 1
I20260927 07:13:18.900118 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.899975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59154)
0927 07:13:18.900040 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:18.900083 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 07:13:18.900431 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.900359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59164)
0927 07:13:18.900394 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:18.900415 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260927 07:13:18.900521 27040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:34927: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
W20260927 07:13:18.951346 27047 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:13:18.951546 27047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:18.951562 27047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:18.951581 27047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:18.952657 27047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:18.952692 27047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:18.952704 27047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:18.952715 27047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:18.952726 27047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:18.954206 27047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:37741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:37187,127.4.174.125:34927,127.4.174.124:37545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab
--principal=kudu/127.4.174.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.125:34927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.174.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:18.954401 27047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:18.954594 27047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:18.959093 27055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:18.959218 27047 server_base.cc:1034] running on GCE node
W20260927 07:13:18.959097 27052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:18.959405 27053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:18.959642 27047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:18.959920 27047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:18.961061 27047 hybrid_clock.cc:648] HybridClock initialized: now 1790493198961048 us; error 28 us; skew 500 ppm
Sep 27 07:13:18 dist-test-slave-zw7p krb5kdc[26959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:18.963402 27047 init.cc:377] Logged in from keytab as kudu/127.4.174.125@KRBTEST.COM (short username kudu)
I20260927 07:13:18.963802 27047 webserver.cc:460] Webserver started at http://127.4.174.125:35067/ using document root <none> and password file <none>
I20260927 07:13:18.964015 27047 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:18.964062 27047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:18.964182 27047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:18.965194 27047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/data/instance:
uuid: "fbef6fb7bcc5400d856c68d640836e7d"
format_stamp: "Formatted at 2026-09-27 07:13:18 on dist-test-slave-zw7p"
server_key: "21d6f53f7b96ce5fcf201ed1dcb4d0ce"
server_key_iv: "97872b285a8579d751ce008249e6cc7d"
server_key_version: "encryptionkey@0"
I20260927 07:13:18.965520 27047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/wal/instance:
uuid: "fbef6fb7bcc5400d856c68d640836e7d"
format_stamp: "Formatted at 2026-09-27 07:13:18 on dist-test-slave-zw7p"
server_key: "21d6f53f7b96ce5fcf201ed1dcb4d0ce"
server_key_iv: "97872b285a8579d751ce008249e6cc7d"
server_key_version: "encryptionkey@0"
I20260927 07:13:18.966768 27047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 07:13:18.967500 27062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:18.967641 27047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:18.967716 27047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/wal
uuid: "fbef6fb7bcc5400d856c68d640836e7d"
format_stamp: "Formatted at 2026-09-27 07:13:18 on dist-test-slave-zw7p"
server_key: "21d6f53f7b96ce5fcf201ed1dcb4d0ce"
server_key_iv: "97872b285a8579d751ce008249e6cc7d"
server_key_version: "encryptionkey@0"
I20260927 07:13:18.967775 27047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:18.968688 27040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } attempt: 2
I20260927 07:13:18.969038 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.968924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59172)
0927 07:13:18.968980 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:18.969010 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260927 07:13:18.969301 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.969225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59174)
0927 07:13:18.969259 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:18.969280 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260927 07:13:18.969375 27040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:34927: Network error: Client connection negotiation failed: client connection to 127.4.174.125:34927: connect: Connection refused (error 111)
I20260927 07:13:18.983173 27047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:18.985316 27047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:18.985463 27047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:18.989053 27047 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.125:34927
I20260927 07:13:18.989100 27114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.125:34927 every 8 connection(s)
I20260927 07:13:18.989419 27047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
I20260927 07:13:18.990367 27115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:18.991948 27115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:18.993815 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 27047
I20260927 07:13:18.993886 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-1/wal/instance
I20260927 07:13:18.994024 4793 external_mini_cluster.cc:1428] Setting key 0bfcdf1551bce475e50a34fbf69efae4
Sep 27 07:13:18 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:13:19.000417 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.992631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.4.174.126:37187)
0927 07:13:18.992694 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:18.992696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:18.992860 (+ 164us) server_negotiation.cc:407] Connection header received
0927 07:13:18.993364 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:18.993366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:18.993380 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:18.993432 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:18.994042 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:18.994539 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:18.995502 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:18.995723 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:18.995778 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:18.997909 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:18.997910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:18.997911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:18.997917 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:18.999168 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:18.999518 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:18.999519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:18.999519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:18.999572 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:18.999775 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:18.999775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:18.999776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:18.999883 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:18.999952 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.000228 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.000328 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 07:13:19.000548 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:18.992578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:37187 (local address 127.0.0.1:49740)
0927 07:13:18.992746 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:18.992782 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:18.993267 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:18.993502 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:18.993505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:18.993739 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:18.993959 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:18.993964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:18.994630 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:18.994631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:18.995425 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:18.995427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:18.995808 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:18.995809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:18.995914 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:18.996384 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:18.996394 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:18.997819 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:18.999238 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:18.999245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:18.999247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:18.999455 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:18.999624 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:18.999624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:18.999625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:18.999723 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:18.999954 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:18.999961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:19.000114 (+ 153us) client_negotiation.cc:769] Sending connection context
0927 07:13:19.000199 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:13:19.000344 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.4.174.124@KRBTEST.COM; defaulting to no policy
I20260927 07:13:19.001024 27115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.124@KRBTEST.COM" created.
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:13:19.008213 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.001236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59182)
0927 07:13:19.001287 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.001297 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:19.001341 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:19.001571 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:19.001571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:19.001668 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:19.001711 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.001712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.002513 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.002513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.003092 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.003094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.003668 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.003669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.003734 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.004031 (+ 297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:19.004035 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:19.005354 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:19.007295 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.007296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.007297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.007461 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.007670 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.007670 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.007670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.007718 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.008000 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:19.008001 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:19.008036 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:13:19.008080 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 07:13:19.008152 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 07:13:19.008324 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.001302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59182 (local address 127.4.174.125:34927)
0927 07:13:19.001457 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.001461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.001475 (+ 14us) server_negotiation.cc:407] Connection header received
0927 07:13:19.001492 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.001493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.001503 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.001559 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.001771 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.002448 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.003163 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.003600 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.003654 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.005440 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.005447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.005447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.005452 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.007186 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.007542 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.007545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.007545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.007607 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.007773 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.007774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.007774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.007941 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.008025 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.008116 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.008227 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260927 07:13:19.008853 27115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:19.009148 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.009041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58576)
0927 07:13:19.009076 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.009126 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260927 07:13:19.009318 27064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
I20260927 07:13:19.009707 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.009611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58592)
0927 07:13:19.009656 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.009685 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260927 07:13:19.009802 27115 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37545: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab.
Entry for principal kudu/127.4.174.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab.
I20260927 07:13:19.017701 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.124:37545
--webserver_interface=127.4.174.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab
--principal=kudu/127.4.174.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:37741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:37187,127.4.174.125:34927,127.4.174.124:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:13:19.059005 27115 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } attempt: 1
I20260927 07:13:19.059495 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.059332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58606)
0927 07:13:19.059399 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.059452 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 07:13:19.059818 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.059741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58622)
0927 07:13:19.059775 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.059800 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260927 07:13:19.059909 27115 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37545: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
I20260927 07:13:19.060498 27040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } attempt: 3
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:13:19.066928 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.060783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59184 (local address 127.4.174.125:34927)
0927 07:13:19.060838 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.060840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.060850 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:13:19.060888 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.060889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.060898 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.060936 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.061252 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.061749 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.062420 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.062700 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.062755 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.064709 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.064710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.064710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.064717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.065903 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.066255 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.066256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.066256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.066301 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.066461 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.066462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.066462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.066604 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.066656 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.066798 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.066859 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 07:13:19.066974 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.060729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59184)
0927 07:13:19.060785 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.060795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:19.060841 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:19.060968 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:19.060969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:19.061120 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:19.061182 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.061184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.061822 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.061824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.062348 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.062350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.062760 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.062762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.062837 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.063230 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:19.063234 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:19.064626 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:19.065977 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.065978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.065979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.066191 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.066351 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.066352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.066352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.066414 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.066658 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:19.066659 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:19.066713 (+ 54us) client_negotiation.cc:769] Sending connection context
0927 07:13:19.066773 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:13:19.066871 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 07:13:19.067389 27040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:19.067703 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.067602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58630)
0927 07:13:19.067644 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.067678 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260927 07:13:19.068014 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.067924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58640)
0927 07:13:19.067968 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.067994 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 07:13:19.068101 27040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37545: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
W20260927 07:13:19.098809 27122 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:13:19.099012 27122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:19.099028 27122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:19.099050 27122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 07:13:19.099237 27040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } attempt: 1
I20260927 07:13:19.099655 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.099505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58646)
0927 07:13:19.099569 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.099618 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 07:13:19.099970 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.099887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58650)
0927 07:13:19.099924 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.099948 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 07:13:19.100073 27040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37545: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
W20260927 07:13:19.100472 27122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:19.100512 27122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:19.100530 27122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:19.100546 27122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:19.100570 27122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:19.102044 27122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:37741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:37187,127.4.174.125:34927,127.4.174.124:37545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab
--principal=kudu/127.4.174.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.124:37545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.174.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:19.102285 27122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:19.102502 27122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:19.107162 27130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:19.107178 27128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:19.107187 27127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:19.107348 27122 server_base.cc:1034] running on GCE node
I20260927 07:13:19.107625 27122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:19.107900 27122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:19.109031 27122 hybrid_clock.cc:648] HybridClock initialized: now 1790493199109017 us; error 26 us; skew 500 ppm
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493199, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:19.111416 27122 init.cc:377] Logged in from keytab as kudu/127.4.174.124@KRBTEST.COM (short username kudu)
I20260927 07:13:19.111829 27122 webserver.cc:460] Webserver started at http://127.4.174.124:41127/ using document root <none> and password file <none>
I20260927 07:13:19.112036 27122 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:19.112083 27122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:19.112202 27122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:19.113194 27122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/data/instance:
uuid: "9e4b9b2692e34e83bfcd90848c1d0167"
format_stamp: "Formatted at 2026-09-27 07:13:19 on dist-test-slave-zw7p"
server_key: "ab49730296649148f05d6b36bcd22ca4"
server_key_iv: "3cfd0105125f096608072a7beb5afa00"
server_key_version: "encryptionkey@0"
I20260927 07:13:19.113512 27122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/wal/instance:
uuid: "9e4b9b2692e34e83bfcd90848c1d0167"
format_stamp: "Formatted at 2026-09-27 07:13:19 on dist-test-slave-zw7p"
server_key: "ab49730296649148f05d6b36bcd22ca4"
server_key_iv: "3cfd0105125f096608072a7beb5afa00"
server_key_version: "encryptionkey@0"
I20260927 07:13:19.114719 27122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 07:13:19.115372 27137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:19.115517 27122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 07:13:19.115605 27122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/wal
uuid: "9e4b9b2692e34e83bfcd90848c1d0167"
format_stamp: "Formatted at 2026-09-27 07:13:19 on dist-test-slave-zw7p"
server_key: "ab49730296649148f05d6b36bcd22ca4"
server_key_iv: "3cfd0105125f096608072a7beb5afa00"
server_key_version: "encryptionkey@0"
I20260927 07:13:19.115669 27122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:19.125149 27122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:19.126817 27122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:19.126940 27122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:19.128057 27115 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } attempt: 2
I20260927 07:13:19.128403 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.128281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58652)
0927 07:13:19.128328 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.128360 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 07:13:19.128767 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.128634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58654)
0927 07:13:19.128691 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.128730 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 07:13:19.128877 27115 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:37545: Network error: Client connection negotiation failed: client connection to 127.4.174.124:37545: connect: Connection refused (error 111)
I20260927 07:13:19.130659 27122 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.124:37545
I20260927 07:13:19.130695 27189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.124:37545 every 8 connection(s)
I20260927 07:13:19.130982 27122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
I20260927 07:13:19.131884 27190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:19.131891 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 27122
I20260927 07:13:19.131961 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492940996973-4793-0/minicluster-data/master-2/wal/instance
I20260927 07:13:19.132128 4793 external_mini_cluster.cc:1428] Setting key 81635928bc4ebb62da77411c96f8068e
I20260927 07:13:19.133996 27190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493199, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:13:19.141616 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.134623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49750 (local address 127.4.174.126:37187)
0927 07:13:19.134663 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.134665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.134818 (+ 153us) server_negotiation.cc:407] Connection header received
0927 07:13:19.134908 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.134909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.134919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.134962 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.135408 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.135854 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.136704 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.136985 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.137033 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.139253 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.139255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.139255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.139263 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.140555 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.140905 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.140906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.140907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.140956 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.141104 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.141105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.141105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.141224 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.141280 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.141458 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.141528 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 07:13:19.141716 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.132591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.4.174.124:37545)
0927 07:13:19.132735 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.132743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.132769 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:13:19.133223 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.133229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.133275 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.133382 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.134088 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.135161 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.135932 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.136340 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.136419 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.138714 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.138721 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.138722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.138728 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.140322 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.140649 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.140652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.140652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.140726 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.140901 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.140901 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.140902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.141168 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.141294 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.141386 (+ 92us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.141515 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260927 07:13:19.141744 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.134591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:37187 (local address 127.0.0.1:49750)
0927 07:13:19.134731 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.134762 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:19.134831 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:19.134985 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:19.134988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:19.135243 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:19.135320 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.135322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.135932 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.135933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.136629 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.136632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.137080 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.137081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.137154 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.137502 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:19.137511 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:19.139170 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:19.140621 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.140628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.140630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.140835 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.141011 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.141012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.141012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.141058 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.141278 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:19.141284 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:19.141346 (+ 62us) client_negotiation.cc:769] Sending connection context
0927 07:13:19.141424 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 07:13:19.141642 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":170,"thread_start_us":76,"threads_started":1}
I20260927 07:13:19.142226 27190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493199, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:13:19.148177 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.142483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59200 (local address 127.4.174.125:34927)
0927 07:13:19.142520 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.142522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.142529 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:13:19.142550 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.142551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.142577 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.142625 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.142816 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.143341 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.144004 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.144249 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.144300 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.145886 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.145887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.145888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.145894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.147242 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.147513 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.147514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.147514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.147564 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.147715 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.147716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.147716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.147829 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.147891 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.148040 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.148100 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 07:13:19.148185 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.142425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:34927 (local address 127.0.0.1:59200)
0927 07:13:19.142463 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.142471 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:19.142500 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:19.142628 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:19.142629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:19.142729 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:19.142753 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.142754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.143403 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.143403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.143939 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.143941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.144308 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.144309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.144360 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.144668 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:19.144672 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:19.145817 (+ 1145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:19.147302 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.147302 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.147303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.147456 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.147616 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.147616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.147617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.147668 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.147904 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:19.147905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:19.147944 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 07:13:19.148010 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 07:13:19.148096 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 07:13:19.148664 27190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:19.148772 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.142398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.4.174.126:37187)
0927 07:13:19.142457 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.142459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.142466 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:13:19.142483 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.142484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.142492 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.142531 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.143033 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.143503 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.144127 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.144311 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.144363 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.146458 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.146459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.146459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.146466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.147707 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.148029 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.148030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.148030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.148091 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.148289 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.148290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.148290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.148418 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.148547 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.148625 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.148698 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493199, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:13:19.154623 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.148955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.4.174.124:37545)
0927 07:13:19.148983 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.148985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.148994 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:13:19.149010 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.149010 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.149017 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.149061 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.149306 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.149930 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.150609 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.150841 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.150898 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.152480 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.152481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.152482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.152489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.153645 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.153936 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.153937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.153938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.153990 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.154163 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.154163 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.154164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.154274 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.154333 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.154469 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.154531 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260927 07:13:19.154723 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.148865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58664)
0927 07:13:19.148907 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.148919 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:19.148958 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:19.149068 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:19.149068 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:19.149211 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:19.149243 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.149245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.149999 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.150000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.150503 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.150505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.150904 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.150905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.150955 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.151268 (+ 313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:19.151273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:19.152401 (+ 1128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:19.153709 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.153710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.153710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.153871 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.154045 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.154045 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.154046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.154102 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.154334 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:19.154334 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:19.154377 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 07:13:19.154451 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 07:13:19.154615 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":46}
I20260927 07:13:19.156142 27190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167: Bootstrap starting.
I20260927 07:13:19.156441 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.149904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59202 (local address 127.4.174.125:34927)
0927 07:13:19.149933 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.149935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.149944 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:13:19.149967 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.149968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.149976 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.150017 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.150438 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.150966 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.151557 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.151785 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.151833 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.153985 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.153987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.153987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.153994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.155466 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.155830 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.155831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.155831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.155875 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.156052 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.156053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.156053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.156137 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.156250 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.156324 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.156384 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:13:19.157052 27190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:19.157341 27190 log.cc:824] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:19.158087 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:19.158142 27190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167: No bootstrap required, opened a new log
I20260927 07:13:19.158995 27014 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "44f09986f4d44f40b014698c1bd69c05"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49756
I20260927 07:13:19.159722 27190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } }
I20260927 07:13:19.159857 27190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4b9b2692e34e83bfcd90848c1d0167, State: Initialized, Role: FOLLOWER
I20260927 07:13:19.159945 27190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } }
I20260927 07:13:19.160234 27190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:19.160210 27196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.160303 27196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 07:13:19.161770 27207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:19.161821 27207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:13:19.162150 27122 master_runner.cc:186] Error getting master registration for 127.4.174.126:37187: OK, instance_id { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" instance_seqno: 1790493198835111 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:13:19.164692 27122 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:13:19.175206 27040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } attempt: 2
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:13:19.181918 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.175549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.4.174.124:37545)
0927 07:13:19.175592 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.175594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.175603 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:13:19.175658 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.175659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.175668 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.175723 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.176026 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.176753 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.177717 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.178012 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.178067 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.179771 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.179772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.179773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.179779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.180957 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.181249 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.181250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.181251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.181303 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.181454 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.181455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.181456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.181582 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.181637 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.181767 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.181831 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:13:19.181926 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.175485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58672)
0927 07:13:19.175550 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.175560 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:19.175597 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:19.175736 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:19.175736 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:19.175911 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:19.175960 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.175962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.176818 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.176819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.177648 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.177650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.178071 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.178072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.178124 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.178479 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:19.178484 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:19.179697 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:19.181035 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.181036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.181037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.181193 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.181353 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.181353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.181354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.181409 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.181637 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:19.181638 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:19.181683 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 07:13:19.181741 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 07:13:19.181830 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 07:13:19.183214 27040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05: Bootstrap starting.
I20260927 07:13:19.183784 27040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:19.184053 27040 log.cc:824] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:19.184705 27040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05: No bootstrap required, opened a new log
I20260927 07:13:19.185664 27040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } }
I20260927 07:13:19.185772 27040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f09986f4d44f40b014698c1bd69c05, State: Initialized, Role: FOLLOWER
I20260927 07:13:19.185849 27040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } }
I20260927 07:13:19.186100 27210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.186162 27014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 07:13:19.186195 27210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:19.186146 27040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:19.187443 27014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:19.188076 27014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } }
I20260927 07:13:19.188789 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44f09986f4d44f40b014698c1bd69c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9e4b9b2692e34e83bfcd90848c1d0167"
I20260927 07:13:19.188916 27164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [term 0 FOLLOWER]: Advancing to term 1
W20260927 07:13:19.189208 27221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:19.189246 27221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:13:19.189805 27164 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44f09986f4d44f40b014698c1bd69c05 in term 1.
I20260927 07:13:19.190012 26989 leader_election.cc:302] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44f09986f4d44f40b014698c1bd69c05, 9e4b9b2692e34e83bfcd90848c1d0167; no voters:
I20260927 07:13:19.190116 27210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:19.190224 27210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [term 1 LEADER]: Becoming Leader. State: Replica: 44f09986f4d44f40b014698c1bd69c05, State: Running, Role: LEADER
I20260927 07:13:19.190305 27210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } }
I20260927 07:13:19.190773 27210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f09986f4d44f40b014698c1bd69c05. Latest consensus state: current_term: 1 leader_uuid: "44f09986f4d44f40b014698c1bd69c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.190835 27210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:19.189909 26972 master_runner.cc:186] Error getting master registration for 127.4.174.125:34927: OK, instance_id { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" instance_seqno: 1790493198988095 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:13:19.192171 26972 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:13:19.188601 27014 leader_election.cc:288] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [CANDIDATE]: Term 1 election: Requested vote from peers fbef6fb7bcc5400d856c68d640836e7d (127.4.174.125:34927), 9e4b9b2692e34e83bfcd90848c1d0167 (127.4.174.124:37545)
I20260927 07:13:19.198683 27225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:19.198904 27225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:19.200785 27164 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [term 1 FOLLOWER]: Refusing update from remote peer 44f09986f4d44f40b014698c1bd69c05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:13:19.201228 27223 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:13:19.202720 27196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f09986f4d44f40b014698c1bd69c05. Latest consensus state: current_term: 1 leader_uuid: "44f09986f4d44f40b014698c1bd69c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.202812 27196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:19.205044 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.193558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.4.174.126:37187)
0927 07:13:19.193604 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.193606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.193937 (+ 331us) server_negotiation.cc:407] Connection header received
0927 07:13:19.194072 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.194073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.194094 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.194121 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.195344 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.196114 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.197688 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.197999 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.198037 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.199769 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.199770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.199771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.199778 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.203037 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.203711 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.203712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.203712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.203784 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.204329 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.204330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.204330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.204483 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.204569 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.204876 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.204955 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 07:13:19.208242 27210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44f09986f4d44f40b014698c1bd69c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.208303 27234 mvcc.cc:205] Tried to move back new op lower bound from 7333860143923236864 to 7333860143884345344. Current Snapshot: MvccSnapshot[applied={T|T < 7333860143923236864}]
I20260927 07:13:19.208340 27210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:19.208421 27196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44f09986f4d44f40b014698c1bd69c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.208506 27196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:19.208555 27233 mvcc.cc:205] Tried to move back new op lower bound from 7333860143923236864 to 7333860143884345344. Current Snapshot: MvccSnapshot[applied={T|T < 7333860143923236864}]
I20260927 07:13:19.210137 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.193740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.4.174.125:34927)
0927 07:13:19.193780 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.193782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.199724 (+ 5942us) server_negotiation.cc:407] Connection header received
0927 07:13:19.199815 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.199816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.199828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.199856 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.201349 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.202091 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.203141 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.203435 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.203484 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.205281 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.205283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.205283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.205290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.207683 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.208946 (+ 1263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.208947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.208947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.209005 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.209269 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.209269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.209270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.209403 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.209495 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.209965 (+ 470us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.210043 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:13:19.211141 27225 catalog_manager.cc:1347] Generated new cluster ID: f7bbbb2753914cf5a676a751ce80204f
I20260927 07:13:19.211193 27225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:19.214900 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.193950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.4.174.124:37545)
0927 07:13:19.193985 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.193987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.205608 (+ 11621us) server_negotiation.cc:407] Connection header received
0927 07:13:19.205742 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.205743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.205754 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.205782 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.206577 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.207256 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.208887 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.209171 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.209209 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.211479 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.211481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.211481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.211489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.213178 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.213799 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.213801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.213801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.213868 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.214168 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.214169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.214170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.214305 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.214369 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.214741 (+ 372us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.214812 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:13:19.220003 27115 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } attempt: 3
Sep 27 07:13:19 dist-test-slave-zw7p krb5kdc[26959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:13:19.229267 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.220250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:37545 (local address 127.0.0.1:58692)
0927 07:13:19.220350 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:19.220360 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:19.220416 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:19.220668 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:19.220669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:19.220833 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:19.220879 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.220881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.221815 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.221816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.222660 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:19.222662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.223226 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.223227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:19.223296 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.223811 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:19.223816 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:19.225354 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:19.227981 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.227982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.227982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.228194 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.228485 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:19.228485 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:19.228486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:19.228557 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:19.228958 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:19.228959 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:19.229017 (+ 58us) client_negotiation.cc:769] Sending connection context
0927 07:13:19.229070 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 07:13:19.229163 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260927 07:13:19.230064 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:19.220490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58692 (local address 127.4.174.124:37545)
0927 07:13:19.220542 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.220543 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.220549 (+ 6us) server_negotiation.cc:407] Connection header received
0927 07:13:19.220581 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.220582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.220591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.220631 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.220973 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.221710 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.222759 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:19.223071 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:19.223139 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:19.225459 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:19.225460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:19.225460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:19.225467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:19.227880 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.228278 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.228279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.228280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.228364 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:19.228624 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:19.228625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:19.228625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:19.228832 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:19.229687 (+ 855us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:19.229758 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 07:13:19.229891 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 07:13:19.230540 27225 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:19.231264 27225 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:19.231784 27115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d: Bootstrap starting.
I20260927 07:13:19.232537 27115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:19.232831 27115 log.cc:824] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:19.233984 27115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d: No bootstrap required, opened a new log
I20260927 07:13:19.236060 27115 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } }
I20260927 07:13:19.236193 27115 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbef6fb7bcc5400d856c68d640836e7d, State: Initialized, Role: FOLLOWER
I20260927 07:13:19.236265 27115 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } }
I20260927 07:13:19.236593 27115 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:19.237859 27088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:19.238744 27088 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [term 1 FOLLOWER]: Refusing update from remote peer 44f09986f4d44f40b014698c1bd69c05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:13:19.238912 27089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44f09986f4d44f40b014698c1bd69c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fbef6fb7bcc5400d856c68d640836e7d"
I20260927 07:13:19.239625 27089 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44f09986f4d44f40b014698c1bd69c05 in term 1.
I20260927 07:13:19.240034 27210 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 07:13:19.242659 27250 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:19.242755 27250 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:13:19.242897 27225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05: Generated new TSK 0
I20260927 07:13:19.243089 27225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:19.243162 27239 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44f09986f4d44f40b014698c1bd69c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.243247 27239 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:19.243287 27239 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f09986f4d44f40b014698c1bd69c05. Latest consensus state: current_term: 1 leader_uuid: "44f09986f4d44f40b014698c1bd69c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.243330 27239 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:19.243353 27239 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44f09986f4d44f40b014698c1bd69c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.243404 27239 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:19.247211 27252 mvcc.cc:205] Tried to move back new op lower bound from 7333860144042557440 to 7333860143884345344. Current Snapshot: MvccSnapshot[applied={T|T < 7333860143923236864}]
I20260927 07:13:19.255084 27223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44f09986f4d44f40b014698c1bd69c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f09986f4d44f40b014698c1bd69c05" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 37187 } } peers { permanent_uuid: "fbef6fb7bcc5400d856c68d640836e7d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 34927 } } peers { permanent_uuid: "9e4b9b2692e34e83bfcd90848c1d0167" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 37545 } } }
I20260927 07:13:19.255195 27223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f09986f4d44f40b014698c1bd69c05 [sys.catalog]: This master's current role is: LEADER
W20260927 07:13:19.265813 27193 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.264352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.4.174.124:37545)
0927 07:13:19.264391 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.264393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.265216 (+ 823us) server_negotiation.cc:407] Connection header received
0927 07:13:19.265310 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.265311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.265322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.265349 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.265750 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58708: BlockingRecv error: recv got EOF from 127.0.0.1:58708 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 07:13:19.266635 27043 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.264691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.4.174.126:37187)
0927 07:13:19.264727 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.264728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.266177 (+ 1449us) server_negotiation.cc:407] Connection header received
0927 07:13:19.266257 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.266258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.266268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.266293 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.266578 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49782: BlockingRecv error: recv got EOF from 127.0.0.1:49782 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 07:13:19.267421 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.264987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.4.174.125:34927)
0927 07:13:19.265028 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.265029 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.266962 (+ 1933us) server_negotiation.cc:407] Connection header received
0927 07:13:19.267069 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.267070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.267080 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.267107 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.267370 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59218: BlockingRecv error: recv got EOF from 127.0.0.1:59218 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 07:13:19.269683 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.268964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59232 (local address 127.4.174.125:34927)
0927 07:13:19.269007 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.269008 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.269237 (+ 229us) server_negotiation.cc:407] Connection header received
0927 07:13:19.269326 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.269327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.269342 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.269368 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.269647 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59232: BlockingRecv error: recv got EOF from 127.0.0.1:59232 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260927 07:13:19.272554 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.271872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.4.174.125:34927)
0927 07:13:19.271900 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.271901 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.272112 (+ 211us) server_negotiation.cc:407] Connection header received
0927 07:13:19.272198 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.272199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.272208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.272233 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.272515 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59238: BlockingRecv error: recv got EOF from 127.0.0.1:59238 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260927 07:13:19.276386 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.275724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59246 (local address 127.4.174.125:34927)
0927 07:13:19.275754 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.275755 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.275942 (+ 187us) server_negotiation.cc:407] Connection header received
0927 07:13:19.276040 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.276041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.276050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.276075 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.276345 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59246: BlockingRecv error: recv got EOF from 127.0.0.1:59246 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260927 07:13:19.282327 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.281578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59250 (local address 127.4.174.125:34927)
0927 07:13:19.281613 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.281615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.281822 (+ 207us) server_negotiation.cc:407] Connection header received
0927 07:13:19.281913 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.281914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.281924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.281950 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.282283 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59250: BlockingRecv error: recv got EOF from 127.0.0.1:59250 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 07:13:19.292382 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.291606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.4.174.125:34927)
0927 07:13:19.291641 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.291642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.291857 (+ 215us) server_negotiation.cc:407] Connection header received
0927 07:13:19.291950 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.291951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.291962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.292002 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.292334 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59264: BlockingRecv error: recv got EOF from 127.0.0.1:59264 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 07:13:19.310891 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.309808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.4.174.125:34927)
0927 07:13:19.309855 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.309857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.310133 (+ 276us) server_negotiation.cc:407] Connection header received
0927 07:13:19.310376 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.310378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.310390 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.310419 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.310828 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59270: BlockingRecv error: recv got EOF from 127.0.0.1:59270 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 07:13:19.345086 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.344329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.4.174.125:34927)
0927 07:13:19.344370 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.344372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.344501 (+ 129us) server_negotiation.cc:407] Connection header received
0927 07:13:19.344625 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.344626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.344638 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.344687 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.345027 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59274: BlockingRecv error: recv got EOF from 127.0.0.1:59274 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 07:13:19.410970 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.410325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.4.174.125:34927)
0927 07:13:19.410414 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.410416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.410431 (+ 15us) server_negotiation.cc:407] Connection header received
0927 07:13:19.410507 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.410508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.410527 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.410623 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.410910 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59286: BlockingRecv error: recv got EOF from 127.0.0.1:59286 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260927 07:13:19.540704 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.540043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59300 (local address 127.4.174.125:34927)
0927 07:13:19.540113 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.540115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.540201 (+ 86us) server_negotiation.cc:407] Connection header received
0927 07:13:19.540274 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.540275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.540286 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.540331 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.540643 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59300: BlockingRecv error: recv got EOF from 127.0.0.1:59300 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 07:13:19.798331 27119 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:19.797755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.4.174.125:34927)
0927 07:13:19.797822 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:19.797824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:19.797833 (+ 9us) server_negotiation.cc:407] Connection header received
0927 07:13:19.797880 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:19.797881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:19.797892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:19.797939 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:19.798267 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59308: BlockingRecv error: recv got EOF from 127.0.0.1:59308 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 07:13:20.162236 27207 catalog_manager.cc:1259] Loaded cluster ID: f7bbbb2753914cf5a676a751ce80204f
I20260927 07:13:20.162307 27207 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167: loading cluster ID for follower catalog manager: success
I20260927 07:13:20.163161 27207 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167: acquiring CA information for follower catalog manager: success
I20260927 07:13:20.163475 27207 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9e4b9b2692e34e83bfcd90848c1d0167: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:13:20.243227 27250 catalog_manager.cc:1259] Loaded cluster ID: f7bbbb2753914cf5a676a751ce80204f
I20260927 07:13:20.243304 27250 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d: loading cluster ID for follower catalog manager: success
I20260927 07:13:20.244122 27250 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d: acquiring CA information for follower catalog manager: success
I20260927 07:13:20.244453 27250 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fbef6fb7bcc5400d856c68d640836e7d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:13:20.313757 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:20.311491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59314 (local address 127.4.174.125:34927)
0927 07:13:20.311624 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:20.311626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:20.311636 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:13:20.311665 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:20.311666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:20.311715 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:20.311785 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:20.311923 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.312413 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.313139 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.313342 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.313395 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:20.313511 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:20.313611 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:20.313662 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:20.313676 (+ 14us) server_negotiation.cc:299] Negotiation successful
0927 07:13:20.313692 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260927 07:13:20.317662 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:20.315450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58710 (local address 127.4.174.124:37545)
0927 07:13:20.315558 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:20.315559 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:20.315566 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:13:20.315599 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:20.315600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:20.315633 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:20.315701 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:20.315776 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.316246 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.316906 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.317115 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.317178 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:20.317302 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:20.317406 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:20.317465 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:20.317550 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 07:13:20.317574 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":46,"threads_started":1}
I20260927 07:13:20.321884 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:20.319260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58724 (local address 127.4.174.124:37545)
0927 07:13:20.319303 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:20.319305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:20.319505 (+ 200us) server_negotiation.cc:407] Connection header received
0927 07:13:20.319562 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:20.319563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:20.319574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:20.319631 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:20.319759 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.320269 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.321161 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.321451 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.321486 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:20.321656 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:20.321686 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:20.321707 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:20.321787 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 07:13:20.321814 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:13:20.321988 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:20.319383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.4.174.125:34927)
0927 07:13:20.319414 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:20.319416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:20.319639 (+ 223us) server_negotiation.cc:407] Connection header received
0927 07:13:20.319715 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:20.319716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:20.319724 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:20.319766 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:20.319944 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.320646 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.321360 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.321559 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.321609 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:20.321733 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:20.321765 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:20.321805 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:20.321900 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 07:13:20.321923 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 07:13:20.322469 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:20.319253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49788 (local address 127.4.174.126:37187)
0927 07:13:20.319365 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:20.319367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:20.319505 (+ 138us) server_negotiation.cc:407] Connection header received
0927 07:13:20.319584 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:20.319585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:20.319634 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:20.319706 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:20.319846 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.320599 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.321575 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:20.321868 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:20.321924 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:20.322148 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:20.322252 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:20.322317 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:20.322368 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 07:13:20.322395 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":56,"threads_started":1}
I20260927 07:13:20.324025 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 26972
I20260927 07:13:20.328903 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 27047
I20260927 07:13:20.333030 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 27122
2026-09-27T07:13:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2668 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:13:20 dist-test-slave-zw7p krb5kdc[27340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:13:20 dist-test-slave-zw7p krb5kdc[27340](info): set up 2 sockets
Sep 27 07:13:20 dist-test-slave-zw7p krb5kdc[27340](info): commencing operation
krb5kdc: starting...
W20260927 07:13:21.286509 4793 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/test-user.keytab.
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.174.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
Entry for principal kudu/127.4.174.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab.
I20260927 07:13:21.396005 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:43653
--webserver_interface=127.4.174.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:43653,127.4.174.125:35045,127.4.174.124:41753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:13:21.474864 27353 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:13:21.475093 27353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:21.475111 27353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:21.475131 27353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:21.476356 27353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:21.476400 27353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:21.476413 27353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:21.476424 27353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:21.476437 27353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:21.477977 27353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:43653,127.4.174.125:35045,127.4.174.124:41753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.126.keytab
--principal=kudu/127.4.174.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.126:43653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.174.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:21.478223 27353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:21.478458 27353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:21.483276 27359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:21.483238 27361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:21.483345 27353 server_base.cc:1034] running on GCE node
W20260927 07:13:21.483212 27358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:21.483747 27353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:21.484033 27353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:21.485216 27353 hybrid_clock.cc:648] HybridClock initialized: now 1790493201485203 us; error 27 us; skew 500 ppm
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:21.487600 27353 init.cc:377] Logged in from keytab as kudu/127.4.174.126@KRBTEST.COM (short username kudu)
I20260927 07:13:21.488059 27353 webserver.cc:460] Webserver started at http://127.4.174.126:35675/ using document root <none> and password file <none>
I20260927 07:13:21.488253 27353 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:21.488299 27353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:21.488417 27353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:21.489456 27353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/data/instance:
uuid: "82f1847e1fc9491aafb80b0be218528f"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "478395355f8ac740e6d87c6775d291ad"
server_key_iv: "dcda85966a3317f4a18c38ffa351b09d"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.489796 27353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/wal/instance:
uuid: "82f1847e1fc9491aafb80b0be218528f"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "478395355f8ac740e6d87c6775d291ad"
server_key_iv: "dcda85966a3317f4a18c38ffa351b09d"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.491146 27353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 07:13:21.491950 27368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:21.492136 27353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 07:13:21.492219 27353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/wal
uuid: "82f1847e1fc9491aafb80b0be218528f"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "478395355f8ac740e6d87c6775d291ad"
server_key_iv: "dcda85966a3317f4a18c38ffa351b09d"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.492281 27353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:21.505903 27353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:21.508083 27353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:21.508219 27353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:21.512045 27353 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.126:43653
I20260927 07:13:21.512077 27420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.126:43653 every 8 connection(s)
I20260927 07:13:21.512413 27353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/data/info.pb
I20260927 07:13:21.513379 27421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:21.514947 27421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:21.520511 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 27353
I20260927 07:13:21.520627 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-0/wal/instance
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:13:21.520850 4793 external_mini_cluster.cc:1428] Setting key 6da9bf1f75a0ed6accf2564d5ff8bb87
I20260927 07:13:21.524434 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.515582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:43653 (local address 127.0.0.1:43358)
0927 07:13:21.515743 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.515781 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.516228 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.516481 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.516484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.516719 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.516901 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.516906 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.517898 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.517898 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.518644 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.518646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.519149 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.519150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.519207 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.519664 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.519671 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.521131 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.522797 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.522805 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.522807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.523087 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.523318 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.523319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.523319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.523413 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.523926 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.523933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.524004 (+ 71us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.524087 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.524217 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260927 07:13:21.524441 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.515636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.4.174.126:43653)
0927 07:13:21.515775 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.515783 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.515857 (+ 74us) server_negotiation.cc:407] Connection header received
0927 07:13:21.516366 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.516368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.516385 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.516471 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.516987 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.517813 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.518710 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.519046 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.519125 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.521215 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.521222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.521223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.521229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.522710 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.523169 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.523171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.523171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.523252 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.523553 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.523554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.523555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.523845 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.523925 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.524134 (+ 209us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.524289 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260927 07:13:21.525235 27421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:21.525557 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.525446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46164)
0927 07:13:21.525492 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.525536 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 07:13:21.525695 27370 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
I20260927 07:13:21.526111 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.526021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46168)
0927 07:13:21.526068 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.526094 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 07:13:21.526247 27421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:35045: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.174.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab.
Entry for principal kudu/127.4.174.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab.
I20260927 07:13:21.545238 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.125:35045
--webserver_interface=127.4.174.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab
--principal=kudu/127.4.174.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:43653,127.4.174.125:35045,127.4.174.124:41753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:13:21.575471 27421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } attempt: 1
I20260927 07:13:21.576030 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.575879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46180)
0927 07:13:21.575942 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.575991 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 07:13:21.576401 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.576310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46182)
0927 07:13:21.576355 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.576382 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 07:13:21.576511 27421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:35045: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
W20260927 07:13:21.624202 27428 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:13:21.624403 27428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:21.624418 27428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:21.624437 27428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:21.625561 27428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:21.625602 27428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:21.625615 27428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:21.625626 27428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:21.625636 27428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:21.627187 27428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:43653,127.4.174.125:35045,127.4.174.124:41753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.125.keytab
--principal=kudu/127.4.174.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.125:35045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.174.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:21.627410 27428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:21.627624 27428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:21.632120 27433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:21.632202 27434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:21.632151 27436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:21.632835 27428 server_base.cc:1034] running on GCE node
I20260927 07:13:21.633091 27428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:21.633414 27428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:21.634593 27428 hybrid_clock.cc:648] HybridClock initialized: now 1790493201634572 us; error 31 us; skew 500 ppm
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:21.637097 27428 init.cc:377] Logged in from keytab as kudu/127.4.174.125@KRBTEST.COM (short username kudu)
I20260927 07:13:21.637490 27428 webserver.cc:460] Webserver started at http://127.4.174.125:41609/ using document root <none> and password file <none>
I20260927 07:13:21.637713 27428 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:21.637761 27428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:21.637893 27428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:21.639006 27428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/data/instance:
uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "71e13586d15df29b1b05cc3a1ac33816"
server_key_iv: "c3ac909d1b4acc90fabf27a4ce9be17e"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.639343 27428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/wal/instance:
uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "71e13586d15df29b1b05cc3a1ac33816"
server_key_iv: "c3ac909d1b4acc90fabf27a4ce9be17e"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.640683 27428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:13:21.641413 27443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:21.641616 27428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 07:13:21.641706 27428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/wal
uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "71e13586d15df29b1b05cc3a1ac33816"
server_key_iv: "c3ac909d1b4acc90fabf27a4ce9be17e"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.641764 27428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:21.644668 27421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } attempt: 2
I20260927 07:13:21.645119 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.644985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46188)
0927 07:13:21.645039 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.645081 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 07:13:21.645430 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.645363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46204)
0927 07:13:21.645396 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.645416 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260927 07:13:21.645514 27421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.125:35045: Network error: Client connection negotiation failed: client connection to 127.4.174.125:35045: connect: Connection refused (error 111)
I20260927 07:13:21.659734 27428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:21.661923 27428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:21.662039 27428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:21.665968 27428 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.125:35045
I20260927 07:13:21.666059 27495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.125:35045 every 8 connection(s)
I20260927 07:13:21.666383 27428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/data/info.pb
I20260927 07:13:21.667419 27496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:21.668880 27496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:21.669900 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 27428
I20260927 07:13:21.669996 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-1/wal/instance
I20260927 07:13:21.670164 4793 external_mini_cluster.cc:1428] Setting key 5bcb1facfb77d8b1312fe61030e9123c
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.174.124@KRBTEST.COM; defaulting to no policy
I20260927 07:13:21.677644 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.669556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43366 (local address 127.4.174.126:43653)
0927 07:13:21.669614 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.669616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.669800 (+ 184us) server_negotiation.cc:407] Connection header received
0927 07:13:21.670313 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.670314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.670330 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.670411 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.671047 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.671667 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.672905 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.673110 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.673168 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.675184 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.675186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.675186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.675194 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.676432 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.676784 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.676785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.676786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.676837 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.677053 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.677054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.677054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.677162 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.677216 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.677475 (+ 259us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.677557 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 07:13:21.677837 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.669503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:43653 (local address 127.0.0.1:43366)
0927 07:13:21.669657 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.669698 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.670204 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.670464 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.670468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.670760 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.670963 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.670967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.671773 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.671774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.672819 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.672822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.673196 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.673197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.673316 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.673783 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.673800 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.675099 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.676506 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.676513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.676516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.676716 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.676892 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.676892 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.676893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.676996 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.677219 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.677226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.677358 (+ 132us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.677443 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.677588 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260927 07:13:21.678323 27496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.174.124@KRBTEST.COM" created.
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:13:21.685056 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.678584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46218)
0927 07:13:21.678753 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.678761 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.678798 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.678915 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.678916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.679022 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.679073 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.679074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.680014 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.680015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.680653 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.680656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.681114 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.681115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.681185 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.681522 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.681527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.682615 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.684158 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.684159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.684159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.684303 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.684473 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.684473 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.684474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.684523 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.684761 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.684762 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.684805 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.684865 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.684958 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260927 07:13:21.685137 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.678636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.4.174.125:35045)
0927 07:13:21.678773 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.678777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.678802 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:13:21.678828 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.678830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.678840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.678899 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.679139 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.679940 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.680735 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.681044 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.681099 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.682690 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.682698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.682698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.682704 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.684082 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.684370 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.684373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.684374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.684421 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.684574 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.684574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.684575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.684704 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.684772 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.684903 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.685026 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260927 07:13:21.685879 27496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:21.686266 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.686127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33452)
0927 07:13:21.686173 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.686217 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 07:13:21.686407 27447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
I20260927 07:13:21.686779 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.686694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33462)
0927 07:13:21.686732 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.686756 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 07:13:21.686916 27496 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:41753: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.174.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab.
Entry for principal kudu/127.4.174.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab.
I20260927 07:13:21.693480 4793 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIteyrz/build/release/bin/kudu
/tmp/dist-test-taskIteyrz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.124:41753
--webserver_interface=127.4.174.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab
--principal=kudu/127.4.174.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.174.84:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.174.126:43653,127.4.174.125:35045,127.4.174.124:41753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:13:21.736178 27496 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } attempt: 1
I20260927 07:13:21.736676 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.736544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33478)
0927 07:13:21.736603 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.736638 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 07:13:21.736666 27421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } attempt: 3
I20260927 07:13:21.737236 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.737126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33484)
0927 07:13:21.737166 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.737205 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260927 07:13:21.737365 27496 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:41753: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:13:21.743633 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.736924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46222)
0927 07:13:21.736989 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.737004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.737074 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.737265 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.737267 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.737451 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.737517 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.737519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.738174 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.738175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.738748 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.738751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.739132 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.739133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.739211 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.739612 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.739617 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.741003 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.742705 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.742706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.742707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.742880 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.743049 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.743050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.743050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.743106 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.743325 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.743326 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.743377 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.743436 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.743536 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 07:13:21.743633 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.737019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46222 (local address 127.4.174.125:35045)
0927 07:13:21.737053 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.737055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.737157 (+ 102us) server_negotiation.cc:407] Connection header received
0927 07:13:21.737182 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.737183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.737194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.737242 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.737598 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.738097 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.738824 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.739066 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.739126 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.741104 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.741106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.741107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.741115 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.742608 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.742942 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.742943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.742944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.742994 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.743156 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.743157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.743158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.743267 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.743330 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.743466 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.743537 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:13:21.744065 27421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:21.744423 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.744299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33498)
0927 07:13:21.744364 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.744396 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 07:13:21.744721 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.744641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33500)
0927 07:13:21.744678 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.744700 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 07:13:21.744801 27421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:41753: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
W20260927 07:13:21.773815 27503 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:13:21.774056 27503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:13:21.774082 27503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:13:21.774107 27503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:13:21.775285 27503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:13:21.775331 27503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:13:21.775349 27503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:13:21.775367 27503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:13:21.775381 27503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:13:21.775947 27421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } attempt: 1
I20260927 07:13:21.776422 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.776254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33516)
0927 07:13:21.776328 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.776383 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 07:13:21.776767 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.776674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33520)
0927 07:13:21.776719 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.776745 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 07:13:21.776851 27421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:41753: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
I20260927 07:13:21.777149 27503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.174.84:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.174.126:43653,127.4.174.125:35045,127.4.174.124:41753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/krb5kdc/kudu_127.4.174.124.keytab
--principal=kudu/127.4.174.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.174.124:41753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.174.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 06:08:55 UTC on ccycloud.quasar-ezlzfr.root.comops.site
build id 4675736
I20260927 07:13:21.777369 27503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:13:21.777542 27503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:13:21.782136 27508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:21.782145 27509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:13:21.782223 27511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:13:21.782478 27503 server_base.cc:1034] running on GCE node
I20260927 07:13:21.782747 27503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:13:21.783074 27503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:13:21.784224 27503 hybrid_clock.cc:648] HybridClock initialized: now 1790493201784211 us; error 26 us; skew 500 ppm
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:13:21.786650 27503 init.cc:377] Logged in from keytab as kudu/127.4.174.124@KRBTEST.COM (short username kudu)
I20260927 07:13:21.787123 27503 webserver.cc:460] Webserver started at http://127.4.174.124:45049/ using document root <none> and password file <none>
I20260927 07:13:21.787344 27503 fs_manager.cc:359] Metadata directory not provided
I20260927 07:13:21.787395 27503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:13:21.787520 27503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:13:21.788570 27503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/data/instance:
uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "e6a78778e09b659a2fe91dd459982bec"
server_key_iv: "548100c229aa0ccab14a4b37bceb54df"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.788918 27503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/wal/instance:
uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "e6a78778e09b659a2fe91dd459982bec"
server_key_iv: "548100c229aa0ccab14a4b37bceb54df"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.790189 27503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 07:13:21.790979 27518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:13:21.791150 27503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 07:13:21.791234 27503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/data,/tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/wal
uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf"
format_stamp: "Formatted at 2026-09-27 07:13:21 on dist-test-slave-zw7p"
server_key: "e6a78778e09b659a2fe91dd459982bec"
server_key_iv: "548100c229aa0ccab14a4b37bceb54df"
server_key_version: "encryptionkey@0"
I20260927 07:13:21.791297 27503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:13:21.805507 27496 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } attempt: 2
I20260927 07:13:21.806061 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.805902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33522)
0927 07:13:21.805968 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.806027 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 07:13:21.806438 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.806365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33534)
0927 07:13:21.806401 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.806422 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 07:13:21.806541 27496 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.174.124:41753: Network error: Client connection negotiation failed: client connection to 127.4.174.124:41753: connect: Connection refused (error 111)
I20260927 07:13:21.809711 27503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:13:21.811498 27503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:13:21.811615 27503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:13:21.815542 27503 rpc_server.cc:307] RPC server started. Bound to: 127.4.174.124:41753
I20260927 07:13:21.815627 27570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.174.124:41753 every 8 connection(s)
I20260927 07:13:21.815981 27503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/data/info.pb
I20260927 07:13:21.816963 27571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:13:21.818274 4793 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIteyrz/build/release/bin/kudu as pid 27503
I20260927 07:13:21.818388 4793 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIteyrz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492940996973-4793-0/minicluster-data/master-2/wal/instance
I20260927 07:13:21.818626 4793 external_mini_cluster.cc:1428] Setting key cc8dad52cab14fb005c337fe73b201c6
I20260927 07:13:21.818708 27571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:13:21.827993 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.819372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.4.174.126:43653)
0927 07:13:21.819434 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.819437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.819581 (+ 144us) server_negotiation.cc:407] Connection header received
0927 07:13:21.819919 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.819920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.819931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.819993 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.820527 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.820996 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.821985 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.822270 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.822335 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.824770 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.824772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.824772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.824779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.826251 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.826783 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.826784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.826785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.826846 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.827371 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.827372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.827372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.827501 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.827569 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.827814 (+ 245us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.827898 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:13:21.828205 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.819116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.4.174.124:41753)
0927 07:13:21.819279 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.819287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.819314 (+ 27us) server_negotiation.cc:407] Connection header received
0927 07:13:21.819817 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.819826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.819859 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.819943 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.820535 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.821453 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.822277 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.822712 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.822772 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.825241 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.825250 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.825250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.825257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.826881 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.827166 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.827168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.827169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.827250 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.827459 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.827459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.827460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.827684 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.827789 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.827940 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.828011 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260927 07:13:21.828337 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.819317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.126:43653 (local address 127.0.0.1:43382)
0927 07:13:21.819460 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.819498 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.819829 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.820009 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.820012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.820271 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.820436 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.820440 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.821067 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.821068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.821908 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.821910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.822381 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.822382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.822517 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.823098 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.823106 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.824687 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.826356 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.826363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.826365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.826708 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.827175 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.827176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.827176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.827295 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.827585 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.827591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.827682 (+ 91us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.827782 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.827951 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260927 07:13:21.828804 27571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.126@KRBTEST.COM
I20260927 07:13:21.835325 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.829032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.125:35045 (local address 127.0.0.1:46228)
0927 07:13:21.829122 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.829133 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.829180 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.829290 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.829291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.829427 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.829473 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.829475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.830273 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.830274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.831063 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.831066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.831438 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.831439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.831487 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.831813 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.831817 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.832963 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.834246 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.834247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.834247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.834393 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.834590 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.834590 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.834590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.834642 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.835075 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.835076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.835109 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.835159 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.835239 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260927 07:13:21.835372 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.829082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46228 (local address 127.4.174.125:35045)
0927 07:13:21.829132 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.829134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.829167 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:13:21.829229 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.829230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.829238 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.829288 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.829536 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.830195 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.831135 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.831371 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.831437 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.833045 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.833046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.833046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.833053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.834174 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.834455 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.834456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.834456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.834508 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.834694 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.834695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.834695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.834800 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.834857 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.835220 (+ 363us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.835286 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 07:13:21.835647 27571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:13:21.835954 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.829079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.4.174.126:43653)
0927 07:13:21.829120 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.829122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.829167 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:13:21.829227 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.829228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.829237 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.829288 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.829765 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.830394 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.831291 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.831546 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.831595 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.833516 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.833517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.833518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.833523 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.834794 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.835062 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.835063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.835063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.835104 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.835281 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.835282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.835282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.835486 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.835624 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.835814 (+ 190us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.835874 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.124@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.174.125@KRBTEST.COM
I20260927 07:13:21.842453 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.835912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33548)
0927 07:13:21.835942 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.835952 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.835987 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.836088 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.836088 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.836205 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.836234 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.836236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.837562 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.837563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.838066 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.838068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.838399 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.838400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.838451 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.838923 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.838927 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.840128 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.841568 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.841569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.841570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.841721 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.841889 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.841889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.841890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.841942 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.842142 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.842143 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.842178 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.842238 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.842340 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260927 07:13:21.842495 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.835952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33548 (local address 127.4.174.124:41753)
0927 07:13:21.836015 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.836016 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.836023 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:13:21.836037 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.836038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.836045 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.836085 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.836355 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.837013 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.838139 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.838333 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.838391 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.840205 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.840205 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.840206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.840211 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.841510 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.841780 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.841781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.841781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.841836 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.841991 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.841991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.841992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.842091 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.842143 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.842262 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.842386 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":49}
I20260927 07:13:21.843133 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.836470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46240 (local address 127.4.174.125:35045)
0927 07:13:21.836501 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.836503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.836510 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:13:21.836567 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.836568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.836575 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.836607 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.837075 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.837495 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.838107 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.838323 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.838371 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.840760 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.840762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.840762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.840769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.841928 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.842211 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.842212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.842212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.842258 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.842421 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.842422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.842422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.842522 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.842809 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.842982 (+ 173us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.843053 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260927 07:13:21.843999 27571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf: Bootstrap starting.
I20260927 07:13:21.844429 4793 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:13:21.844894 27571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:21.845171 27571 log.cc:824] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:21.845916 27571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf: No bootstrap required, opened a new log
I20260927 07:13:21.845187 27395 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "82f1847e1fc9491aafb80b0be218528f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43398
I20260927 07:13:21.847397 27571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } }
I20260927 07:13:21.847523 27571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd128c99ea74e9fb05c0f7e7a6997bf, State: Initialized, Role: FOLLOWER
I20260927 07:13:21.847604 27571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } }
I20260927 07:13:21.847899 27571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:21.847883 27577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.847991 27577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [sys.catalog]: This master's current role is: FOLLOWER
W20260927 07:13:21.849452 27588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:21.849509 27588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:13:21.849776 27503 master_runner.cc:186] Error getting master registration for 127.4.174.126:43653: OK, instance_id { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" instance_seqno: 1790493201510912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:13:21.851471 27503 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:13:21.851964 27421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } attempt: 2
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.126@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:13:21.859063 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.852196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33560)
0927 07:13:21.852233 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.852263 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.852305 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.852443 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.852444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.852615 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.852666 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.852668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.853542 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.853543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.854354 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.854358 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.854819 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.854820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.854879 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.855319 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.855324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.856769 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.858127 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.858128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.858129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.858286 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.858461 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.858462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.858462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.858517 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.858776 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.858777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.858824 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.858883 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.858973 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 07:13:21.859063 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.852241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33560 (local address 127.4.174.124:41753)
0927 07:13:21.852278 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.852279 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.852290 (+ 11us) server_negotiation.cc:407] Connection header received
0927 07:13:21.852365 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.852366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.852374 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.852426 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.852730 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.853464 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.854431 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.854751 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.854813 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.856840 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.856841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.856842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.856848 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.858053 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.858355 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.858356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.858356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.858408 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.858591 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.858592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.858592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.858717 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.858774 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.858913 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.858974 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:13:21.860332 27421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f: Bootstrap starting.
I20260927 07:13:21.860934 27421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:21.861181 27421 log.cc:824] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:21.861846 27421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f: No bootstrap required, opened a new log
I20260927 07:13:21.862870 27421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } }
I20260927 07:13:21.862973 27421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f1847e1fc9491aafb80b0be218528f, State: Initialized, Role: FOLLOWER
I20260927 07:13:21.863034 27421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } }
I20260927 07:13:21.863265 27591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.863317 27421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:21.863358 27591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:21.863372 27395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 07:13:21.863442 27395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:21.864187 27395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } }
I20260927 07:13:21.864658 27545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82f1847e1fc9491aafb80b0be218528f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf"
I20260927 07:13:21.864810 27545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:21.865634 27545 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f1847e1fc9491aafb80b0be218528f in term 1.
I20260927 07:13:21.865846 27372 leader_election.cc:302] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bd128c99ea74e9fb05c0f7e7a6997bf, 82f1847e1fc9491aafb80b0be218528f; no voters:
W20260927 07:13:21.865900 27602 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:21.865939 27602 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:13:21.865970 27591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:13:21.866086 27591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [term 1 LEADER]: Becoming Leader. State: Replica: 82f1847e1fc9491aafb80b0be218528f, State: Running, Role: LEADER
I20260927 07:13:21.866164 27591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } }
I20260927 07:13:21.866637 27591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82f1847e1fc9491aafb80b0be218528f. Latest consensus state: current_term: 1 leader_uuid: "82f1847e1fc9491aafb80b0be218528f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.866729 27591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:21.867153 27395 leader_election.cc:288] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [CANDIDATE]: Term 1 election: Requested vote from peers 5a65f898c1404f47aacb6a9fe6fd1c2d (127.4.174.125:35045), 1bd128c99ea74e9fb05c0f7e7a6997bf (127.4.174.124:41753)
I20260927 07:13:21.865593 27353 master_runner.cc:186] Error getting master registration for 127.4.174.125:35045: OK, instance_id { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" instance_seqno: 1790493201664833 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:13:21.867815 27353 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:13:21.871145 27606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:13:21.871335 27606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:13:21.873375 27545 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [term 1 FOLLOWER]: Refusing update from remote peer 82f1847e1fc9491aafb80b0be218528f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:13:21.873768 27604 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:13:21.875736 27577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82f1847e1fc9491aafb80b0be218528f. Latest consensus state: current_term: 1 leader_uuid: "82f1847e1fc9491aafb80b0be218528f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.875823 27577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:21.877231 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.868276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.4.174.124:41753)
0927 07:13:21.868319 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.868321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.868942 (+ 621us) server_negotiation.cc:407] Connection header received
0927 07:13:21.869050 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.869051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.869062 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.869118 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.869858 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.870625 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.871636 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.871919 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.871992 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.874025 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.874026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.874026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.874046 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.875637 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.876189 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.876190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.876191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.876249 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.876543 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.876544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.876544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.876686 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.876757 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.877063 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.877144 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 07:13:21.877274 27604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82f1847e1fc9491aafb80b0be218528f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.877370 27604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:21.878037 27577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82f1847e1fc9491aafb80b0be218528f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.878098 27577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:21.879568 27615 mvcc.cc:205] Tried to move back new op lower bound from 7333860154871070720 to 7333860154844655616. Current Snapshot: MvccSnapshot[applied={T|T < 7333860154871070720}]
I20260927 07:13:21.880401 27606 catalog_manager.cc:1347] Generated new cluster ID: bc837f13396148669a6e9656988c30bd
I20260927 07:13:21.880451 27606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:13:21.885283 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.868311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.4.174.126:43653)
0927 07:13:21.868354 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.868355 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.877861 (+ 9506us) server_negotiation.cc:407] Connection header received
0927 07:13:21.877986 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.877987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.878009 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.878037 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.878941 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.879732 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.881161 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.881475 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.881538 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.882901 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.882902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.882902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.882909 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.884278 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.884617 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.884618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.884619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.884662 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.884805 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.884806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.884806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.884911 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.884983 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.885135 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.885198 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:13:21.889408 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.868671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.4.174.125:35045)
0927 07:13:21.868722 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.868724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.877971 (+ 9247us) server_negotiation.cc:407] Connection header received
0927 07:13:21.878058 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.878059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.878069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.878097 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.883114 (+ 5017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.883831 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.884765 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.885058 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.885111 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.886751 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.886753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.886754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.886761 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.888242 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.888660 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.888660 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.888661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.888713 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.888879 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.888897 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.888898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.889010 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.889084 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.889241 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.889310 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 07:13:21.897719 27496 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } attempt: 3
Sep 27 07:13:21 dist-test-slave-zw7p krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790493201, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.174.125@KRBTEST.COM for kudu/127.4.174.124@KRBTEST.COM
I20260927 07:13:21.905846 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.897982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.174.124:41753 (local address 127.0.0.1:33584)
0927 07:13:21.898029 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 07:13:21.898038 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 07:13:21.898071 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:13:21.898268 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:13:21.898268 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:13:21.898452 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:13:21.898498 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.898500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.899440 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.899442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.900253 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:13:21.900256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.900708 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.900709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:13:21.900769 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.901252 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:13:21.901257 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:13:21.903111 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:13:21.904802 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.904803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.904804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.905016 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.905222 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:13:21.905223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:13:21.905223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:13:21.905282 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:13:21.905535 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:13:21.905536 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:13:21.905583 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 07:13:21.905644 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 07:13:21.905742 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 07:13:21.906231 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:21.898039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33584 (local address 127.4.174.124:41753)
0927 07:13:21.898140 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.898142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.898150 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:13:21.898172 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.898173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.898183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.898225 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.898600 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.899361 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.900344 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:21.900646 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:21.900713 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:21.903197 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:13:21.903198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:13:21.903199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:13:21.903205 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:13:21.904715 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.905096 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.905097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.905097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.905162 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:13:21.905355 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:13:21.905356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:13:21.905356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:13:21.905479 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:13:21.905535 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:21.906008 (+ 473us) server_negotiation.cc:299] Negotiation successful
0927 07:13:21.906114 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 07:13:21.907913 27496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: Bootstrap starting.
I20260927 07:13:21.908142 27606 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:13:21.908622 27496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: Neither blocks nor log segments found. Creating new log.
I20260927 07:13:21.908915 27496 log.cc:824] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:13:21.909170 27606 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:13:21.909660 27496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: No bootstrap required, opened a new log
I20260927 07:13:21.911657 27496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } }
I20260927 07:13:21.911806 27496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a65f898c1404f47aacb6a9fe6fd1c2d, State: Initialized, Role: FOLLOWER
I20260927 07:13:21.911913 27496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } }
I20260927 07:13:21.912731 27496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:13:21.912878 27470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82f1847e1fc9491aafb80b0be218528f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d"
I20260927 07:13:21.912995 27470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:13:21.913828 27470 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f1847e1fc9491aafb80b0be218528f in term 1.
W20260927 07:13:21.914055 27449 tablet.cc:2369] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:13:21.914124 27469 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [term 1 FOLLOWER]: Refusing update from remote peer 82f1847e1fc9491aafb80b0be218528f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:13:21.914443 27604 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:13:21.916465 27606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f: Generated new TSK 0
I20260927 07:13:21.916699 27606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:13:21.917044 27635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82f1847e1fc9491aafb80b0be218528f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.917135 27635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:21.917178 27635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82f1847e1fc9491aafb80b0be218528f. Latest consensus state: current_term: 1 leader_uuid: "82f1847e1fc9491aafb80b0be218528f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.917214 27635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:21.917238 27635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82f1847e1fc9491aafb80b0be218528f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.917270 27635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:13:21.918088 27591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82f1847e1fc9491aafb80b0be218528f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f1847e1fc9491aafb80b0be218528f" member_type: VOTER last_known_addr { host: "127.4.174.126" port: 43653 } } peers { permanent_uuid: "5a65f898c1404f47aacb6a9fe6fd1c2d" member_type: VOTER last_known_addr { host: "127.4.174.125" port: 35045 } } peers { permanent_uuid: "1bd128c99ea74e9fb05c0f7e7a6997bf" member_type: VOTER last_known_addr { host: "127.4.174.124" port: 41753 } } }
I20260927 07:13:21.918174 27591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f1847e1fc9491aafb80b0be218528f [sys.catalog]: This master's current role is: LEADER
I20260927 07:13:21.918329 27633 mvcc.cc:205] Tried to move back new op lower bound from 7333860155009646592 to 7333860154844655616. Current Snapshot: MvccSnapshot[applied={T|T < 7333860154871070720}]
W20260927 07:13:21.919090 27631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:13:21.919148 27631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 07:13:21.946432 27574 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.944740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.4.174.124:41753)
0927 07:13:21.944786 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.944788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.945782 (+ 994us) server_negotiation.cc:407] Connection header received
0927 07:13:21.945888 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.945889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.945899 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.945927 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.946361 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33598: BlockingRecv error: recv got EOF from 127.0.0.1:33598 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 07:13:21.947369 27424 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.945151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.4.174.126:43653)
0927 07:13:21.945191 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.945192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.946858 (+ 1666us) server_negotiation.cc:407] Connection header received
0927 07:13:21.946950 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.946951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.946962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.946991 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.947291 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43414: BlockingRecv error: recv got EOF from 127.0.0.1:43414 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 07:13:21.948247 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.945464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.4.174.125:35045)
0927 07:13:21.945502 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.945504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.947731 (+ 2227us) server_negotiation.cc:407] Connection header received
0927 07:13:21.947816 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.947817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.947827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.947871 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.948187 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46264: BlockingRecv error: recv got EOF from 127.0.0.1:46264 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 07:13:21.950860 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.950042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46272 (local address 127.4.174.125:35045)
0927 07:13:21.950078 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.950080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.950265 (+ 185us) server_negotiation.cc:407] Connection header received
0927 07:13:21.950388 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.950389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.950409 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.950453 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.950814 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46272: BlockingRecv error: recv got EOF from 127.0.0.1:46272 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 07:13:21.954054 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.953269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.4.174.125:35045)
0927 07:13:21.953310 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.953311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.953569 (+ 258us) server_negotiation.cc:407] Connection header received
0927 07:13:21.953661 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.953662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.953672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.953700 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.954007 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46276: BlockingRecv error: recv got EOF from 127.0.0.1:46276 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260927 07:13:21.957959 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.957247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46292 (local address 127.4.174.125:35045)
0927 07:13:21.957279 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.957281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.957509 (+ 228us) server_negotiation.cc:407] Connection header received
0927 07:13:21.957592 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.957593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.957602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.957627 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.957917 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46292: BlockingRecv error: recv got EOF from 127.0.0.1:46292 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260927 07:13:21.964208 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.963322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46294 (local address 127.4.174.125:35045)
0927 07:13:21.963385 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.963387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.963634 (+ 247us) server_negotiation.cc:407] Connection header received
0927 07:13:21.963731 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.963732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.963743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.963771 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.964120 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46294: BlockingRecv error: recv got EOF from 127.0.0.1:46294 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 07:13:21.974635 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.973687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.4.174.125:35045)
0927 07:13:21.973727 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.973729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.973995 (+ 266us) server_negotiation.cc:407] Connection header received
0927 07:13:21.974097 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.974098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.974110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.974138 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.974529 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46310: BlockingRecv error: recv got EOF from 127.0.0.1:46310 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 07:13:21.993078 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:21.992132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.4.174.125:35045)
0927 07:13:21.992177 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:21.992179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:21.992489 (+ 310us) server_negotiation.cc:407] Connection header received
0927 07:13:21.992596 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:21.992597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:21.992609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:21.992637 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:21.993015 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46318: BlockingRecv error: recv got EOF from 127.0.0.1:46318 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 07:13:22.027267 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:22.026586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.4.174.125:35045)
0927 07:13:22.026630 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:22.026633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:22.026742 (+ 109us) server_negotiation.cc:407] Connection header received
0927 07:13:22.026808 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:22.026808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:22.026820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:22.026879 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:22.027203 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46324: BlockingRecv error: recv got EOF from 127.0.0.1:46324 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260927 07:13:22.092976 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:22.092354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.4.174.125:35045)
0927 07:13:22.092450 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:22.092452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:22.092488 (+ 36us) server_negotiation.cc:407] Connection header received
0927 07:13:22.092548 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:22.092549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:22.092559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:22.092607 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:22.092915 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46326: BlockingRecv error: recv got EOF from 127.0.0.1:46326 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260927 07:13:22.222916 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:22.222214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46336 (local address 127.4.174.125:35045)
0927 07:13:22.222281 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:22.222283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:22.222376 (+ 93us) server_negotiation.cc:407] Connection header received
0927 07:13:22.222456 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:22.222457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:22.222469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:22.222510 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:22.222867 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46336: BlockingRecv error: recv got EOF from 127.0.0.1:46336 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260927 07:13:22.480890 27500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:13:22.480193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.4.174.125:35045)
0927 07:13:22.480252 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:22.480254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:22.480262 (+ 8us) server_negotiation.cc:407] Connection header received
0927 07:13:22.480289 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:22.480289 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:22.480299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:22.480369 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:13:22.480795 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46348: BlockingRecv error: recv got EOF from 127.0.0.1:46348 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 07:13:22.849960 27588 catalog_manager.cc:1259] Loaded cluster ID: bc837f13396148669a6e9656988c30bd
I20260927 07:13:22.850036 27588 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf: loading cluster ID for follower catalog manager: success
I20260927 07:13:22.850871 27588 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf: acquiring CA information for follower catalog manager: success
I20260927 07:13:22.851171 27588 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1bd128c99ea74e9fb05c0f7e7a6997bf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:13:22.919533 27631 catalog_manager.cc:1259] Loaded cluster ID: bc837f13396148669a6e9656988c30bd
I20260927 07:13:22.919605 27631 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: loading cluster ID for follower catalog manager: success
I20260927 07:13:22.920809 27631 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: acquiring CA information for follower catalog manager: success
I20260927 07:13:22.921135 27631 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5a65f898c1404f47aacb6a9fe6fd1c2d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:13:22.996498 27704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:22.993947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.4.174.125:35045)
0927 07:13:22.994044 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:22.994046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:22.994056 (+ 10us) server_negotiation.cc:407] Connection header received
0927 07:13:22.994155 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:22.994157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:22.994207 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:22.994291 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:22.994447 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:22.995001 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:22.995786 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:22.996013 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:22.996075 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:22.996205 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:22.996315 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:22.996377 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:22.996394 (+ 17us) server_negotiation.cc:299] Negotiation successful
0927 07:13:22.996412 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":51,"threads_started":1}
I20260927 07:13:23.000474 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:22.998151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33606 (local address 127.4.174.124:41753)
0927 07:13:22.998271 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:22.998273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:22.998280 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:13:22.998322 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:22.998323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:22.998357 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:22.998423 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:22.998530 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:22.999054 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:22.999754 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:22.999955 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:23.000006 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:23.000150 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:23.000255 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:23.000319 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:23.000372 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 07:13:23.000391 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":49,"threads_started":1}
I20260927 07:13:23.004601 27704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:23.001982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.4.174.125:35045)
0927 07:13:23.002019 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:23.002020 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:23.002098 (+ 78us) server_negotiation.cc:407] Connection header received
0927 07:13:23.002189 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:23.002190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:23.002200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:23.002238 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:23.002349 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:23.003085 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:23.003882 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:23.004151 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:23.004209 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:23.004353 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:23.004386 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:23.004431 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:23.004511 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 07:13:23.004532 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 07:13:23.005049 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:23.002033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.4.174.124:41753)
0927 07:13:23.002074 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:23.002076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:23.002480 (+ 404us) server_negotiation.cc:407] Connection header received
0927 07:13:23.002500 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:23.002500 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:23.002510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:23.002549 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:23.002718 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:23.003374 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:23.004242 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:23.004503 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:23.004559 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:23.004764 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:23.004794 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:23.004831 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:23.004968 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 07:13:23.004986 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 07:13:23.005087 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:13:23.002109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43426 (local address 127.4.174.126:43653)
0927 07:13:23.002282 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0927 07:13:23.002283 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 07:13:23.002290 (+ 7us) server_negotiation.cc:407] Connection header received
0927 07:13:23.002323 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:13:23.002324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:13:23.002356 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:13:23.002421 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:13:23.002546 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:23.003061 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:23.003964 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:13:23.004250 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:13:23.004297 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:13:23.004633 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:13:23.004851 (+ 218us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:13:23.004902 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 07:13:23.004977 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 07:13:23.005004 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":47,"threads_started":1}
I20260927 07:13:23.006474 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 27353
I20260927 07:13:23.011413 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 27428
I20260927 07:13:23.015476 4793 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIteyrz/build/release/bin/kudu with pid 27503
2026-09-27T07:13:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2683 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5351 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (262025 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST